I would like to let you know that I am not quite sure what I am doing, I have an idea and I am trying to look for answers, sorry for any non-sense I could say.
I am trying to come out with some design for a FBP (Flow Based Programming) platform but I want some extra features on the implementation.
My vague thoughts
I want every single operation on the software to be a Node on the Graph, even multiple, sum or which ever one (still thinking why but eventually the same platform is used for program itself). Also, I want to make a Smart systems running in the connection between those Nodes, so I could scale the parallelism or take some decisions on the execution of specific program.
Right now, I am thinking on create a GenServer for every single Node (remember it is any computation) but I am not sure if I shouldn’t use GenServer for this since the latency will be involve.
Also, how I could load dynamically into the compiled code new code so the new Nodes becomes available and the software could start using it.
I need your feedbacks
I would like to know the implications of me using new processes for anything, my understand of the BEAM and ErLang is to little for understand what I am asking for (hopefully I will get there). Should I use GenServer or go the most low level implementation of dealing with processes? What could be the duplicated work I will be doing?
The Smart Graph Connections will take a look to the throughput and try to be smart about it, either by using more resources of the computer or maybe auto scale where you could do more parallel process at specific bottleneck on the execution.
Loading new graph node dynamically, the idea is that you would load specific Node implementation or replace the existing one, propagate that to other VMs when it is done correctly so they are aware of the new code without you having to do the same process in each VM.
Anyway, probably my thoughts are too high level and vague but this is something that I am trying to figure out myself what I am trying to do even, it makes sense to me on my head, I need to figure out if makes sense on the real world.