Oh you’re right, I played with it a bit and Enum.chunk_every
is quite slow because the implementation is generic for all enumerables. If I write a specialized version just for lists instead, then I get similar timings for PelemayFp
, Task.async_stream
, and pmap
(with batches of 12,000 items).
I noticed a small detail, the benchmark code is using a module constant for the list:
@list Enum.to_list(1..100000)
This may affect the timing a bit since constants aren’t subject to garbage collection and don’t get copied when they’re sent from one process to another, unlike variable data.