@tsukit, the example in the link can easily illustrate word count in parallel. I had the problem of loading GeoData into a database before starting an application. I had tried to use GenServers with poolboy, but it required a lot of manual tweaks to get it to optimized working. So I tried to search for alternatives and found Flow that had used Genstages and sufficed my needs at that time. At the moment I don’t have that code with me.
The below thread has some important needs for understanding Flow:
File.stream!("path/to/some/file")
|> Flow.from_enumerable()
|> Flow.flat_map(&String.split(&1, " "))
|> Flow.partition()
|> Flow.reduce(fn -> %{} end, fn word, acc ->
Map.update(acc, word, 1, & &1 + 1)
end)
|> Enum.to_list()
The magic here is the :min_demand and :max_demand, it uses the number of cores by default, but you can increase it or decrease it based on your needs.
