I’m not familiar with diffsinger, or the problem space, but I have used visual programming a fair bit over the years for various tasks. So I think I can say something about that part at least. Indeed I have earlier mused that many Elixir programs, with their modules and connections visualized with a Mermaid graph, could look very familiar to visual programming.
I would not choose visual programming for the UI alone. It can be very elegant with small programs, but for big programs it can easily turn into real life visual spaghetti code. At which point making components, color coding and or framing entire areas of code with what they actually do is a good idea. Then you have bigger components which again hides detailed complexity and allow easy overview.
My experience is that visual programming have some strong points:
- Isolation, reuse and making components. This is just trivial, and the reuse is real.
- Refactoring. Again, often just trivial cut and paste and move around.
- Parallelism. You can make independent parallel paths of visual code and you will get programs that runs that code in parallel. (Depending on actual hardware, and not all visual programming languages do this equally well). Back in the day I did FPGA programming using Labview, and it was like painting code onto actual hardware.
- UI. You see your program flow very easily. Depending on the language programming concepts like loops can be less elegant or just not possible at all.
- Debugging. Issues are often highlighted at the exact spot or module. Data comes in, but data doesn’t come out. Making a type breaking connection is usually impossible and thus found before even trying to compile. (Again, depending on language).
Like Elixir visual programming is also about transforming data, through visible steps and with connections between independent modules. Like functional programming the variables don’t change so if you want to reuse the value of a variable from earlier in the graph just make a connection to it at that stage.
The bypass caching & incremental generation you mention is a necessity during development of big graphs, or any minor change will end up triggering entire graph recalculations.
I don’t see why the internal architecture of components would make a visual UI for composing them any harder? The components should be independent, and with explicit inputs and outputs only, so what goes on inside is just their business. Indeed many visual languages make it easy to use different languages inside of components to better suit the components task. For composition the input and outputs, with types and data shapes, should be enough.
