Flowex is a Flow-Based Programming framework for Elixir.
If you have never heard about the library I would recommend you read the library Readme and this post on Medium.
If you are familiar with Flowex check the latest version, the library has been improved significantly since the first release.
Recently I’ve published post Flow-Based REST API with Flowex and Plug which has an example of designing approach with FBP paradigm and benchmarks that shed some light on Elixir GenStage performance.
So, check it out, share your thoughts and ask questions.
I’m curious, if there can only be one error_pipe :error_handler and its order does not matter, why not add it as an option to use Flowex.Pipeline like use Flowex.Pipeline error_pipe: :error_handler or even as a full argument to make it more explicit?
Hi @OvermindDL1
“Physically” error_pipe is always the last pipe in a pipeline. And components are ordered in the way you define them, so it is natural to place error_pipe after all defined pipes.
The pipeline is up and running OK. For curious, I killed the pipeline process, but I can see only Producer process is back, but not the others. Is it what you expected or do I miss any?
Below screen shots are before and after killing the pipeline from :observer.
Red lined Flows are what I created and the other processes are created by Flowex pipeline.
Hi @ikim
Seems like you kill the supervisor of the pipeline. This is very unexpected case. Current implemetation start supervisor with producer only and then adds stages one by one and subscribe to each other. So it will fail to restart all the stages after supervisor crash.
But I think this can be changed. May I ask you to create an issue with the example?
Thanks!
I just found out about Flowex and FBP from one of your Youtube videos that led me to your thread on this forum.
I am new to Elixir and programming in general but this idea of FBP sounds very exciting to me as I am a musician so am curious if I am understanding the core foundation of what Flowex/FBP offers?
Would the following example of writing a song be a good analogy?
Instead of focusing on writing one melody on a piano and ONLY on that melody sequentially, Flowex allows a programmer to have the freedom to write not only the piano but the guitar parts, then perhaps the drums, then back to the piano, then the drums, simultaneously?
If so, I will be your most ardent fan and tester as frankly, I’ve found most languages extremely painful and uninspiring (for me at least) with exception of Elixir (obviously) so FBP seems much more suited to my brain where I’m accustomed to jamming on different aspects of a song/project to complete the required goal.
Hi, @synthslave!
Your analogy is very interesting. I can imagine two ways of using Flowex that can be interpreted in musician way):
There are several pipelines, one for each instrument. IPs (informational packets) are the notes. You pass the notes to these pipelines and they are played in parallel.
There is one pipeline where each component is a note. And each component consists of several processes. IPs are the instruments. You pass the instruments into to pipeline and enjoy music.
It sounds cool, but the main problem in both cases - how to synchronize all the instruments. BEAM VM under the hood does not evaluate all the processes in parallel, but frequently switch between them. You can’t be sure that all the notes are played simultaniously. So, in both cases, you need a mechanism of synchronization.
The good news is that a real program is not a music and it is ok if some parts are not synced well.
And I agree with you that FBP is very convienient approach to right software.
What’s the appeal of having a different process do each step, rather than have a pool of processes that execute all the steps in sequence? I had a look through the examples, but I didn’t find a clear answer.
Hi @antonmi !
Thanks for elaborating on the music analogy. That helps put it into more context. Just as an FYI, here are two technologies that actually synchronize the notes for different instruments that are very commonplace in music production/performance:
As an FYI, after learning of Flowex and FBP, I actually reached out to J Paul Morrison and he was very kind and appreciative. Not sure if you ever communicated with him but he’s very cool. Am excited to learn/practice Flowex/FBP.
@dom
Here’s a video from Paul Morrison himself where he explains the origins of FBP:
Hi @synthslave!
Thanks for the information.
Yeap, know about J Paul Morrison and have read his excellent book “Flow-Based Programming”, but haven’t communicated with him.
