Wait for last task or job to finish

Questions / Help
1

I have a bunch of tasks or jobs I can run in parallel. What’s the best to run them and wait for the last one to finish?
I know I can use spawn or Task to run them.

However, how do I wait for the last task to finish?

1 Like
2

You can use something like:

[task1, task2, task3] |> Task.yield_many()

From docs: https://hexdocs.pm/elixir/Task.html#yield_many/2

2 Likes
3

And if I were to use spawn instead of Task?

4

Send a message back to the parent process, then have that process wait to receive every response:

parent = self()
refs = Enum.map([1,2,3], fn n ->
  ref = make_ref()
  spawn_link(fn -> Process.sleep(n * 1000); send(parent, {:done, ref}) end)
  ref
end)
Enum.each(refs, fn ref ->
  receive do
    {:done, ref} -> :ok
  end
end)
1 Like
5

how would I know if a task has failed?

6

The parent process (and all tasks) will crash together. That’s the point of using spawn_link.

If you don’t want that, you can start the tasks via a supervisor and use Process.monitor to detect crashes. That’s how Tasks work:

7

In my case, it might not crash, but return a bad result and thus it’ll have to be re-run again until it returns a good result. how would I re-run it? should I check for a result being good inside “spawn_link” and then re-run it if needed?

9

also, why did you pick yield and not await??

10

Yeah that would work. If you need bounded retries you can do something recursive like this:

  def call_with_retries(args, attempts \\ 5) do
    case call_some_api(args) do
      {:ok, result} ->
        {:ok, result}
      {:error, _} when attempts > 0 ->
        call_with_retries(args, attempts - 1)
      {:error, reason} ->
        {:error, reason}
    end
  end
2 Likes
11

my question wasn’t how to create a function, it was:

In my case, it might not crash, but return a bad result and thus it’ll have to be re-run again until it returns a good result. how would I re-run it? should I check for a result being good inside “spawn_link” and then re-run it if needed?

12

My response to that was “yeah that would work”.

13

hi! I’m having the same problem. Do you have a sample code for this?

Thanks!