How long does that take?
Task.yield_many/2
has a default timeout of 5 secs - maybe that isn’t enough time for Ecto to finish all the queries.
The issue is the tasks are behaving randomly.
I wouldn’t necessarily expect Ecto + RDBMS to behave in a deterministic fashion (order of work finished). So the apparent randomness wouldn’t be too surprising if the timeout is too short.
The other advantage of Task.async_stream/5
is that the level of concurrency can controlled with :max_concurrency
(it defaults to the number of online cores).
There is also Task.Supervisor.async_stream_nolink/6
.