Hi,
I just finished watching another inspiring presentation about upcoming Broadway library and the problems with complex pipelines that @josevalim talked about couldn’t omit my full attention.
That part start at 35:29 https://www.youtube.com/watch?v=Fq9hpl7xbWM&t=35m29s
Most of the GenStage articles out there are describing scenario with a one single Producer. That is a gap in a knowledge.
I particularly never implemented scenario with multiple producers yet, but the time is coming
If you guys ever done I would like to hear your experience, particularly about:
- data processing locality - how do you ensure that events with a same
foo_id
are going through the same consumer? - graceful shutdown - that
Terminator
process magic that can delay the tree shutdown until all other process are done with their jobs.
Examples and samples will be much appreciated!