Firing off supervised tasks and waiting on the results

Questions / Help
1

I have GenServer where I am pulling multiple database records, doing some processing and then updating the records. This job runs every 10 minutes.

Those records are processed with Enum.map and don’t depend on each other, if one crashes (bad data), I don’t want it to affect the others. However I want to block on these tasks being returned because if the GenServer job runs again it will mess up the data.

The code below is similar to what I am doing. However testing this in iex returns instantly. How can I block until I get the results?

children = [
  {Task.Supervisor, name: MyApp.TaskSupervisor}
]

Supervisor.start_link(children, strategy: :one_for_one)

[1,2,3,4,5,6]
|> Enum.map(fn sleep_time -> Task.Supervisor.async(MyApp.TaskSupervisor,
      fn ->
        :timer.sleep(sleep_time)
      end)
    end)
|> Enum.map(&Task.await(&1))
2

The argument to :timer.sleep is the amount of time to sleep in milliseconds. Your loop will sleep for at most 6 milliseconds and then return. It is blocking, it’s just too short to see.

As far as your goal goes however, Task.Supervisor.Async and Task.await will still crash the parent process when they die, so you may want to pick some of the other Task functions that won’t do that.

3 Likes
3

Jeez, dumb mistake on my part. That explains the instant return.

Tried start_child, async_nolink they both crash the supervisor.

4

Have a look at Task.Supervisor.async_stream/6 (and Task.Supervisor.async_stream_nolink/6) and a Task.Supervisor.async_nolink/5 example (and explanation).

2 Likes
5

Thanks I’ll look into this.