Is it wise to default to always using Flow when using Broadway?

Questions / Help
,
1

Greetings, comrades.

I have thoroughly gone through both the RabbitMQ Elixir tutorials (RabbitMQ tutorial - "Hello world!" | RabbitMQ) and the book “Concurrent Data Processing in Elixir”. GenStage is very cool, Flow is very cool and Broadway is very cool.

I have a question for veterans who are well-experienced in both Flow and Broadway. Both are capable of constructing GenStage pipelines but, as far as I can tell, Flow allows for “longer” pipelines with more intricate branching. If using a message broker, it seems like a theoretically “ideal” architecture which would afford maximum future extensibility would be one where Broadway acts as a gateway for Flow.

My question, in short, would be “can Broadway construct pipelines of the same arbitrary length and complexity as Flow”? If not, then could it be said that it is theoretically ideal (when not certain of long-term project requirements) to always use Flow when using Broadway? As I understand, Broadway is concerned about the AMQP side of things, whereas Flow is more concerned with making GenStages do magic concurrency things.

Thanks!

P.S. Flow is really really fun. I am going to try to apply it to caint (Use streams · Issue #5 · peaceful-james/caint · GitHub)

1 Like
2

I’m sorry I don’t have the answer. I’ve never even used Broadway or Flow or GenStage. But I very much enjoyed the reviews of Caint.

1 Like
3

Necromancy sneak attack! I have been using these libs for a few days and am now 90% sure that I am right to default to always using flow with broadway, to orchestrate the genstages that process amqp events.

2 Likes
4

Wow, just the project I was looking for in my attempt to bring a “suite of libs for total localized and translated apps”. Great work.

@paulsabou is also in for LLM Po-files. Great dev I once worked with. Maybe both projects can learn from each other: GitHub - paulsabou/gettext_llm: Elixir Gettext LLM based translation library

1 Like
5

Thanks @BartOtten

1 Like