tsukit

tsukit

Best practice for doing fanout async calls

I have a GenServer defined as a module X whose each instance can be instantiated with a different parameter. My app creates multiple instances of this GenServer. When serving an coming request, the main flow needs to send a message to each instance of X asynchronously, so each instance can process the call in parallel. The main flow waits to get all the responses before moving on.

I am not sure what’s the best way to do this. Currently this is what I am doing.

defmodule X do
  ...
  def handle_cast({:doit, caller, p1, ..}, state) do
    ..
    send(caller, ...)
    {:noreply, state}
  end
  ..
  def do_it_async(server, caller, p1, ...) do
    GenServer.cast(server, {:doit, caller, p1, ..})
  end
end

defmodule Other do
  ..
  def handleRequest(...) do
    servers_list = ..
    servers_list |> Enum.each(&X.do_it_async(&1, self(), ..)
    results = collect_results([], length(servers_list), 0)
    ..
  end

  defp collect_results(results, count, received) when count == received, do: {:ok, results}
  defp collect_results(results, count, received) do
    receive do
      {:x_result, ..} ->
        collect_results([{..} | results], count, received + 1)
    after
      5000 -> :timeout
    end
  end

This is obviously very tedious and error-prone. What is the recommended / common way to do this kind of thing in Elixir?

I am aware of Task.async_stream and I could provide a sync version of the API. However, unless I misunderstood it, I don’t think that’s the right way to do it because it would spawn so many processes just to call the API and wait for the result. Each X instance is already a process.

Most Liked

entone

entone

Registry also provides a nice way to do pub-sub without having to introduce GenStage or back pressure or anything like that, Registry — Elixir v1.20.2

pmangalakader

pmangalakader

Hi @tsukit,

If your intention is just parallel processing or computation along with throttling, then you can take a look at Elixir Flow.

If it is something different, then a little more elaborate would help understand the problem you’re trying to solve.

Thanks.

Where Next?

Popular in Questions Top

vertexbuffer
Hello, can anybody help here..? I have a list of players and I what to delete an element, but every for loop the list is reverting to ori...
New
beno
I will often find my self writing things similar to: case some_value do nil -> something() "" -> something() _ -> somethi...
New
qwerescape
Is there a way to get the call stack or stack trace at any point in the code? Not from exceptions, but an expression that returns how the...
New
minhajuddin
I have seen a lot of code which picks the first element from a list using Enum.at(0) instead of List.first. Is there a reason why people ...
New
belgoros
I’m not a pro in using Regex and can’t figure out why the following behaviour happens, especially if we take into account the difference ...
New
nobody
Hi! In PHP: $_SERVER[‘SERVER_ADDR’] - in Elixir? Searched the docs for ip address and the web, no good results. Thanks!
New
fayddelight
I tried installing elixir 1.11.2 erlang 23.3.4 via asdf in my zsh shell. Enabled the versions locally and globally. When I list them ...
New
komlanvi
Hi everyone, I was playing with phoenix liveView but I run into an issue. I have a form and want to validate each input text when the te...
New
Brian
What is the proper way to load a module from a file in to IEX? In the python world, doing something like this pretty standard: from ....
New
vegabook
I’m brand new to Phoenix and I have stripped one of the demo applications to the bone. I just want to get an svg up on the screen. Here i...
New

Other popular topics Top

marius95
Hello everyone, I try to use an Javascript Event Handler in my root.html.leex file. Therefore I created a function in the app.js file: ...
New
Darmani72
If I have a post route which an argument: post /my_post_route/:my_param1, MyController.my_post_handler How would get the post params ...
New
electic
Hi, I am new to Elixir. I am trying to use the DateTime component to insert a date into MySQL however the there seems to be no way to fo...
New
pmjoe
I have a relationship of love and hate with Elixir. Lots of things are just absolutely right, but there are some things that are kind of ...
New
ashish173
I am using Ecto timestamps with postgres, I can see the timestamps() use the :naive_dateime but for my use case I wanted to store the ti...
New
boundedvariable
I am going through the kafka architecture. All the features what the kafka is providing are already in Erlang. I would like hear your opi...
New
openscript
Hello! Sorry for this astonishing simple question, but I’m really stuck. I try to set up the intellij-elixir plugin, but I don’t know ho...
New
hariharasudhan94
Lets say I have map like this fetching from my database %{"_id" => #BSON.ObjectId<58eb1a7a9ad169198c3dXXXX>, "email" => ...
New
JorisKok
I have a server on AWS, and was running a load test using artillery. When looking at the Phoenix dashboard I see the Ports going to 100% ...
New
jononomo
For some reason my phoenix channels are working for me in my local dev environment, but as soon as I deploy via Docker, I get a 403 error...
New

We're in Beta

About us Mission Statement