wolf4earth

wolf4earth

What are you building your event-driven elixir systems on?

There are a number of ways to go about building an event-driven system in Elixir and which exactly you’re using depends on the needs of the application you’re building. This thread is meant to get a “lay of the land” for which tools the community likes to reach for.

To get us started here are some options I’ve seen mentioned in the wild:

What is your weapon of choice and why? Are you beginning with something simple and replace it with something else when the scale demands it? How do you structure your application when building a system like this and why?

Most Liked

MrDoops

MrDoops

“It depends.” When it comes to messaging in Elixir there’s so many ways to approach it so there’s a suite of options.

Registry/PubSub are good for transient consumers with weaker guarantees. A liveview or channel can subscribe and the pubsub semantics deliver the message but if that Liveview session gets killed / disconnected: not a big deal. So a good tool for natural OTP process <> process delivery - just handle_info callbacks and a simple interface for topics.

Genstage/Broadway are good for high throughput data pipelines and in a classic CQRS model are usually more seen in either command handling or projection/read-model construction.

I would also look at Eventstore and some of the Commanded tooling on top. In most workflows where pubsub is used you still need a persistent model and eventstore gives you a way to persist each event and deliver to n subscribers. The persistent stream interface has more going on than Phoenix PubSub, but it lets you revive a state machine by reading past events and rebuilding state.

After you have a persistent model of some kind, you might want to push that event to a messaging system or queue like RabbitMQ/SQS/Kakfa/etc either for integration to another system or to buffer some expensive data pipeline operation. Broadway in particular builds on Genstage to provide a high level tool to control for load and throughput such as to accumulate a set of “things to insert_all” into a database and prevent overload against potential bottleneck or protected resource like your reporting database.

Finally you might have those integration events that go out to external systems - this is where something like webhooks, Kafka, RabbitMQ streams, or some kind of high throughput messaging system with pubsub semantics, good subscriber guarantees and widespread client library support is good.

So all of the above when it depends.

katafrakt

katafrakt

If I were to build the whole app around events and reacting to them (which is a great way to build an app!), I would probably use Commanded on top of PostgreSQL (personal preference, I don’t like Event Store).

For just having an event bus to do some things reactively (and async) – event_bus package you mentioned looks great. Although I don’t have real experience with it.

For just organizing the codebase, you can just as well use a synchronous/inline event dispatcher, as described here or (shameless plug) here.

Where Next?

Popular in Discussions Top

Donovan
Hello everyone, I’m so glad to have discovered this awesome community. Thanks for creating it! This is my second post, and apologies for...
New
Fl4m3Ph03n1x
Background A few days ago I was listening to The future of Elixir from Elixir Talks, with Dave Thomas (@pragdave ) and Brian Mitchell. I...
New
WolfDan
After doing a port from a c++ library to my project in phoenix I’ve seen that I need a faster way to run this algorithm and I found this ...
New
praveenperera
How We Replaced React with Phoenix By: Thought Bot
New
lucaong
Hello Elixir and Nerves community, I have been working for a while on an open-source embedded key-value database for Elixir, that I call...
230 13924 124
New
lorenzo
Hey everone! I created a prototype for my app using Nodejs for the api. But the framework I chose wasnt great (in general theresnt any g...
New
AstonJ
Please see the new poll here: Which code editor or IDE do you use? (Poll) (2022 Edition) It’s been a while since we first asked this, I...
208 31142 143
New
eteeselink
Hi all, In the last days, two things happened: A blog post titled “They might never tell you it’s broken” made the rounds. It’s about ...
New
joeerl
I’m playing with Elixir - It’s fun. I think @rvirding does give Elixir courses these days. Re: files and database - when I given Erlang ...
New
sergio
Kind of like when jquery came out, it was super necessary. Existing drag and drop libraries have a bunch of baggage to support old browse...
New

Other popular topics Top

aadeshere1
I have a another noob question about loop. Since elixir is immutable, while loop is not directly possible. total = 10 while total != 0 ...
New
New
JeremM34
Hello, how can I check the Phoenix version ? Thanks !
New
Fl4m3Ph03n1x
About me? ( if you have nothing better to do than reading about some random guy in the internet :stuck_out_tongue: ) Hello all, this is ...
New
msaraiva
Surface is an experimental library built on top of Phoenix LiveView and its new LiveComponent API that aims to provide a more declarative...
564 43622 214
New
pmjoe
I have a relationship of love and hate with Elixir. Lots of things are just absolutely right, but there are some things that are kind of ...
New
bsollish-terakeet
Credo is smart enough to check for (something like) this: assert length(the_list) == 0 with this response: Checking if an enum is empt...
New
shijith.k
I am trying to start a new phoenix project with elixir 1.9, but mix phx.new does not work. It says that ** (Mix) The task "phx.new" could...
New
axelson
This post is a wiki (feel free to hit the edit button near the bottom right of this post to add your own changes!) This post collects co...
239 47930 226
New
openscript
Hello! Sorry for this astonishing simple question, but I’m really stuck. I try to set up the intellij-elixir plugin, but I don’t know ho...
New

We're in Beta

About us Mission Statement