What and how to use? GenServer? GenStage?

Hi!
I’m learning Elixir and I find it amazing. At the moment I’m trying to understand and learn to properly use concurrency.

I have the following problem: I have multiple (thousands) of xml files I need to process and export specific information to csv file. I thought about the following structure:

  1. Process that holds the paths to the files (PathDispatcher)
  2. Multiple processes that ask PathDispatcher for paths, process the file and send the result to
  3. ResultGatherer which saves the results to csv file as they come.

I am able to write 1 and 3 using GenServer (is it the best option?), but can’t figure how to write 2. Could you give me any hints how to deal with that?

Cheers,
r00s

There is Task for point 2. It is a simple abstraction over GenServer that might do the trick (process the file and send the result to). If your problem is more complex, there is GenStage and Flow.

In particular, I would look for Task.async_stream.

2 Likes