ElasticFlow - create distributed flow based programs

This is at a working proof of concept stage. I’m interested in any thoughts and feedback toward the design. A Flow use case came up at work and I got through it on a single machine by splitting the work up… but would have saved a lot of time with a few more resources so decided to take a crack at this library now.

Running the example it comes with is the best way to see what it does I think but for the most part; it’s some infra to make it easy to distribute the work of an input you’d send to flow evenly across a cluster (using flow to do that as well), track it’s progress and aggregate results on the master node as they come in.

6 Likes