elbasti
Waiting for multiple tasks in a genserver
I have a genserver that wants to spawn multiple tasks (in this case multiple api calls to different services) and wait for all of them to return, and then process all of the responses simultaneously.
I was originally going to use Task.await_many/2 to well, wait for all of the tasks to return, but this comment in the documentation made me doubt:
It is not recommended to
awaitlong-running tasks inside an OTP behaviour such asGenServer
The recommendation is to instead match on the message coming from a task inside your GenServer.handle_info/2
The docs have a really nice example for monitoring a single task, but how would you write that handle_info function to wait for multiple tasks to return (especially things like handling timeouts for the individual tasks)?
Most Liked
vfsoraki
To build upon the proposed solution, I’d say you can batch :do_all_the_things in one separate process. Ignoring task supervisor, this is a rough code.
def handle_call(:do_all, _from, state) do
urls = […]
gen_server_pid = self()
batch_pid = spawn(fn ->
tasks = Enum.map(urls, START_TASK_FOR_ONE_URL)
result = Task.await_many(tasks)
# You can include a kind of ID here too to distinguish batches
send(gen_server_pid, {:batch_result, result})
end)
# Keep batch_pid if you need it
{:reply, :ok, update_state(state, batch_pid)}
end
def handle_info({:batch_result, result}, state) do
# Process results
{:noreply, state}
end
You get the idea. You start a separate process that spawns processes to process each mini task, then gathers all results and sends them to gen server in one message.
al2o3cr
What’s responsible for deciding to make those requests? If it’s a request from outside doing something like GenServer.call(pid, :make_all_the_requests), what should happen if another make_all_requests comes in while the first one is still processing?
One straightforward alternative would be to have the GenServer launch a single Task which then launches the others and does an await_all.
stevensonmt
I would assume Task.Supervisor.async_stream_nolink is the starting point. Interesting question and I hope someone can provide a concrete example for you.
Popular in Questions
Other popular topics
Categories:
Sub Categories:
Forums
Popular Tags
- #ecto
- #liveview
- #troubleshooting
- #learning-elixir
- #deployment
- #library
- #erlang
- #testing
- #genserver
- #mix
- #absinthe
- #remote-other
- #otp
- #plug
- #how-to-question
- #macros
- #postgres
- #channels
- #elixirconf
- #exunit
- #discussion
- #code-sync
- #javascript
- #podcasts
- #onsite
- #dialyzer
- #docker
- #authentication
- #umbrella
- #full-time-contract
- #podcasts-by-brainlid
- #ecto-query
- #elixir-ls
- #phoenix_html
- #iex
- #blog-post
- #graphql
- #genstage
- #ai
- #websockets
- #supervisor
- #elixirconf-us
- #advent-of-code
- #distillery
- #processes
- #forms
- #api
- #metaprogramming
- #hex
- #security









