Utilizing 100% CPU through spawning multiple processes to execute parts of the loop in parallel is obviously better and turns out to be much faster than a simple single process loop for huge enumerations. A code snippet for people to use if needed. Cheers!
1 Like
Looks neat! One nit, though, there are unnecessary bindings on line 5 (followers =), 14 (task =), 15 (followers_chunk =) and 21 (followers =) that may trigger compiler warnings on unused variable. They can be removed, since last expressions are always returned.
Yeah I forgot removing those parts, I did face those compiler warnings. I have updated the gist, cheers!
Try [element | acc] rather than acc ++ [element]. Appending (++) in a reduce loop means you’re copying the whole output list on every iteration. If order matters, you can reverse the list at the end anyway.
1 Like
Awesome tip, thanks!
