Hello.

I’m designing an application where multiple node have to communicate but I’m not using the Erlang distribution only raw tcp messaging. And I want to set backpressure for the incoming messages to avoid overflow of the system and be able to scale with multiple processors.

I ended up to define something like this using Broadway:

• Listener which spawns a process for each connection
• Each connection spawns a Broadway pipeline with a producer which queue the tcp messages and a given number of processors to deal with the request and send back data

This is working fine, but I’m wondering if it’s a good design to have n Broadway Pipeline or how will it be better to have a single pipeline for the entire app (listener) and provide more processors to scale.

Thanks you