makeitrein

makeitrein

Happy Saturday, Elixir Forum - I’m exploring the Commanded event sourcing library for a potential weekend project. A coworker of mine mentioned that a common concurrency issue in event sourcing is having command handlers write multiple events to a stream when only one written event is valid.

For example, two “close bank account” commands are fired off at the same time, and the handler processes both. Thus, the handler writes two “closed bank accounts” events to the event stream. Only one “closed bank accounts” event should have been written.

Our discussion made me realize that I might not know what I’m getting into when using Event Sourcing - are there other potential concurrency problems that should I be wary of? I see that Commanded has a section related to “Command dispatch consistency guarantee” - does strong consistency protect against common concurrency problems?

Showing Posts 1 to 10

bottlenecked

bottlenecked

Hi, not sure how exactly Commanded is implementing aggregates, but I’d expect they are using processes- which means that barring some grievous design error (either on their part or yours) that should not be a situation you will ever find yourself in, since processes can only ever process one message at a time. So in your example even if two ‘close account’ commands reach your process, the second one should fail the validation because the first should have been fully processed (command → validation → domain event → apply → store and publish)* before ever the next message is consumed from the process’s inbox.

Having said that, be sure to understand the tradeoffs when going the ES way for some part of your system- unless you really need it, you may be trading too much development effort for benefits you’re not using

*I may be getting the order wrong because I’m not familiar with the Commanded’s flavor of ES

benwilson512

benwilson512

Author of Craft GraphQL APIs in Elixir with Absinthe

Command handlers dont get to write to the event stream, they aren’t allowed. Command handlers have to ask an aggregate to actually do the command, and those aggregates serialize commands, meaning they run just one at a time. If a particular event sourcing library can’t guarantee serialized command handling for an aggregate, then it is a seriously flawed library. To my knowledge, Commanded does all of the right things there.

This does of course mean you need to choose your aggregates wisely, and understand how to run operations that involve multiple aggregates. To use your example, a bank account is a fine choice of aggregate, and the aggregate’s job will be to ensure that things like double closes don’t happen. Where things get tricky is something like a transer, wherein you want to move some funds from one aggregate to another. This tends to push things in a “TransferRequested” “TransferAccepted” style event log, which I actually think is pretty solid.

I’m not sure that these are common. If a library says it’s doing event sourcing, but it doesn’t provide a way to do serialized event handling around specific topics or aggregates, I’m not really sure it’s doing event sourcing. Certainly not CQRS.

madlep

madlep

The serialisation of data writes you raise should be fine, as others have mentioned.

The big concurrency issue that trips people up though is eventual consistency. if you’re writing a web front end for a todo list (as an example), and the user adds a new todo, there may be a lag in between the time the event is written, and an event handler running to build a projection for the front end to query. That means the user might not see the new todo appear when they click “save” - which can be weird UX. Commanded has ways to specify how that is handled, but it means you need to think about that up front to understand the consistency/performance trade offs.

madlep

madlep

Where “should be fine” is dependent on how you’ve modelled your aggregates and validation and events and things.

Event sourcing gives you a lot of tools for handling those scenarios, but it means you need to understand your data and your business logic. It often means more work and more moving pieces than an equivalent application written as a regular CRUD app. For a lot of apps though, that trade off makes sense.

Particularly anything that needs an audit trail or guarantees around what is changed and how and when. If you’re dealing with money, you might want event sourcing. If writing a blog or a todo list, you might not.

makeitrein

makeitrein OP

Thx @bottlenecked, @benwilson512, and @madlep - good orientation before I dive too deep into code land.

Definitely recognize that ES requires a fairly high degree of data modeling and coding precision that punishes slip-ups… but I think I’m willing to pay that cost (famous last words).

I’d like to build undo and history functionality for a particular model, and event sourcing seems like a reasonable paradigm to support this. Most of the other attempts that I’ve had in the past coding this functionality always felt a bit janky - here’s hoping that ES will provide a better path forward.

Still deciding on whether to ES the entire application, or just the areas that need it. It feels like an all-or-nothing proposition, and it’d be a bit awkward to mix multiple styles at once.

bottlenecked

bottlenecked

Hey there, if you have a good case for it then by all means go ahead- the warning was to make someone that thinks ES ‘is cool’ rethink their decision carefully.

As for the all in- there are many Greg Young videos talking about ES, and in one or two of those he warns ‘please don’t tell me you’ve built an event sourced system’. His argument is that there is probably just one part of a system that needs it, and it’s OK to pay the cost of admission there- otherwise you will be entering a world of unnecessary complexity.

So… be careful :slight_smile:

domvas

domvas

I ask myself the same question few months ago about doing an all ES application, because I wanted one style for all the system and I really wanted to some ES / CQRS stuff with append- only store and everything. And the more I thought about my project and the more I learn about ES/CQRS, the more I started to feel that having ES everywhere was not a good solution. But I wanted to do it anyway! Then I followed the commanded tutorial (which I highly recommend you to do) and well it opens my eyes: for a lot of part of my project, the amount of work will be huge for not so much benefits and with eventual consistency on top.
At the end, most of the project is classic and one part is some kind of homemade ES, homemade because I don’t need the full power of commanded for now.
And I learn another thing: RGPD destroy ES (and statistics) (if you really comply to rgpd…) :slight_smile:

Good luck with your project, ES is great!

andrejsm

andrejsm

Did you mean GDPR?

domvas

domvas

Yep, I mean GDPR but I used the french term, sorry for that…

saverio-kantox

saverio-kantox

GDPR does not destroy EventSourcing. You just need to source separately the business data (which is not strictly tied to a person) and the personal data (which you can put in separate silos and clean up when needed).

I’m not saying it’s easy, but it’s doable without affecting the event-sourced aggregations.

Where Next? Top

Trending in Questions Top

Blokh
Hey guys, I’ve got a huge CSV ( around 10 GB ) that needs to be processed hourly Do you guys have any suggestions what is the best prac...
New
kszambelanczyk
Hello! Could someone please give me a help/sample code, how to delete a file from s3 using waffle/waffle_ecto from Phoenix app. I creat...
New
Onor.io
I have what I’ve heard referred to as a “lookup table” in my database. This is a way of assigning codes to common values. One common lo...
New
Trolleger
What approach to take when sending live updates to “random” users Hi! I have a question, I have a little chat app, and when I create a DM...
New
RemyXRenard
I’m seeing that a list inside a Kino.DataTable will be interpreted as a charlist, even if the Kino.configure() is set to charlists: :as_l...
New
matt-savvy
Anyone here using Honeybadger? My Honeybadger account is being overwhelmed with noise from some bots. Seeing a lot of Bandit.HTTPError...
New
samoloth
Hi, I’ve just set up an application with ash_authentication. There is only magic link strategy for now, so there is no confirmation add o...
New

Other Trending Topics Top

mudasobwa
I am happy to introduce the very α version of the new programming language compiled to BEAM. Welcome Cure. It has literally three kille...
New
garrison
Hobbes is a low-level distributed database for the Elixir programming language. Hobbes provides a simple, safe, and scalable storage lay...
New
mcass19
ExRatatui lets you cook up rich terminal UIs in Elixir, powered by Rust’s ratatui via Rustler NIFs. Build interactive terminal applicatio...
New
Damirados
Hello everyone. After busy few months I am happy to announce v0.1.0 of Emerge & Solve. They are GUI (Emerge) and State management (S...
New
netoum
Corex is an accessible, unstyled UI component library for Phoenix that integrates Zag.js state machines using Vanilla JavaScript and Live...
New
wintermeyer
There are three potential reasons for members of this forum to have a look at https://vutuv.de You are tired or annoyed of LinkedIn. Yo...
New

We're in Beta

About us Mission Statement

Options

Thread Display Mode




Thread Preview

Skip Thread Previews