type1fool

type1fool

Must RFC - Event Sourcing, simplified?

After discussing some of my ideas around Event Sourcing at last night’s Elixir Happy Hour in Austin (highly recommend), I spent some time distilling things into a new package called Must.

Event Sourcing provides many technical and business benefits. However, the pitch often starts with technical details that overwhelm newcomers and lead to squabbles about consistency and complexity. I believe there is an opportunity to make Event Sourcing concepts more approachable for teams and leaders who are apprehensive about unfamiliar and intimidating aspects of this technique.

The Goal

Must should make it easier for engineering teams to build Event Sourced systems, with or without prior experience.

Motivation

As much as I feel comfortable using Event Sourcing & CQRS, much of the conversation and tooling feels overly academic. It’s a tough sell.

I also believe the concepts, which are often relegated to finance and highly regulated domains, could become more mainstream given a few modifications to the terminology and technical implementations.

While this package does not use the same verbiage and implementation, it takes inspiration from the first-principles design of the Verbs library for PHP.

The Ask

I am looking for feedback about this approach to ES from anyone who has interest. Please take a look at the documentation and source code. I have a short list of starter questions to get the conversation going:

  • Will the Must protocols & behaviour(s) make it easy/easier to spin up Event Sourcing systems?
  • Do the words used in the protocols make sense, particularly when they are composed together?
  • Which storage adapters are most desired?
  • Is this approach doomed to fail?
  • Do you have interest in contributing in general or to particular features?

It’s very early, and the picture may not be clear yet how these ideas will pan out. I hope this discussion will help validate the utility of this approach and lead to a strong set of tools for bringing event-sourced systems to life.

Resources

Most Liked

LostKobrakai

LostKobrakai

To me GitHub - CargoSense/fable: Your events have a story to tell. · GitHub is still the most simple introduction one can have to event sourcing on elixir. It easily retrofits onto a regular ecto architecture, it’s 1000 loc you can read in like 15 minutes and most of it is simple boilerplate - if you can do phoenix with ecto you can understand the fable codebase. As with e.g. commanded you want to skip ProcessManager. Comparatively commanded feels like a kubernetes in terms of complexity in setup - useful, with lots of the mentioned sidequests.

I personally am a big fan of discovering concepts from first principle, so maybe that’s why I like the approach of fable so much. It shows you how to store decisions with events and updating an aggregate without needing to name those things. Once you got to see how such a system works then you can start giving the pieces names like the commands, aggregates, projections whatnot. Once the basic idea is explained you can start getting into all the additional ideas necessary to cover the sharp edges a more naive implementation might not care for or run into.

To turn this into the direction of Must. I like the idea of a more generalizable system, but it imo does suffer from e.g. expecting “Commands” where a user might not know what to make of the word command. At least for the context of teaching I’d personally stay with fable and maybe then update to Must if it eventually delivers a more featureful step beyond fable before needing to take the step to e.g. commanded.

dimitarvp

dimitarvp

This is way too true. In my entire career I’ve never once heard anyone explain it an approachable manner and at one point I stopped looking. It felt like Haskell fanatics explaining how monads are the Universe’s hidden ether essence.

I am not going to go question by question, just going to give you a high-level take: I work in finance. Having audit logs for pretty much anything is not just table stakes, it’s mandatory or you can get in a huge trouble; auditors don’t much care if you can’t explain $1.37 or $98700.62.

However, all the BS terms (sorry not sorry) like “aggregates” really don’t help.

As a start, I believe the whole thing should be very dev-centric i.e. “aggregates are periodic snapshots of all the accumulated events”. Friggin plain and simple, every good dev will get it immediately. (And I might have gotten the term and the explanation wrong which would be both funny and also prove my point perfectly.)

Storage adapters: TigerBeetle (financial ledger, might not apply to everything) and PostgreSQL. Rest can wait.

Approach is doomed to fail if it continues being snobby hand-wavy cult requiring too much homework from people who came to be sold on an idea. NOT to get a new university degree.

tcoopman

tcoopman

I know but it still triggered me a bit :slight_smile:

I agree, and it’s something that is overlooked. But I do like to start at the why first. If we understand the why and can agree when it’s a good idea to use it, then we can have a look at how - which of course should be factored in the decision as well.
If the how is complex then even a good why might not be enough.

I think it’s also important to reflect on the fact that in essence eventsourcing is a different concept than state based systems. We often compare complexity against those systems that we already know and forget it took us (me at least) some time to learn those systems as well. So that means we should give it a fair chance (if the why matters to us).

I’m not going to explain the how here - Idon’t have time to do a decent explanation currently.
I’m also ignoring CQRS completely, I know, it just not that important for the essence of eventsourcing. I have some good diagrams on both of these things actually from courses that I gave in the past but I don’t have good written text. Maybe someday I’ll share that more broadly :slight_smile:

Where Next?

Popular in RFCs Top

wolf4earth
Every protocol you’ve shipped assumes the spec doesn’t change during the connection. This one doesn’t. Code is typed, composable, content...
New
KristerV
I got fed up with removing unused aliases manually so made a package for it: GitHub - KristerV/remove_unused_ex: Remove unused aliases an...
New
senconscious
EctoJuno is a package that provides parsing, validation and applying sorting parameters for your ecto queries on models. Currently sortin...
New
WyvernoDragon
I’ve been working on pgRx: a query engine that uses BEAM lightweight processes instead of PostgreSQL’s fork-per-connection model. The nu...
New
zachallaun
Note: There are a few folks I’d really love to hear from, time permitting. Pinging in case the title isn’t catchy enough :slight_smile: @...
New
jarlah
Hi! I have recently created, after having tried to get in touch with the creator of excontainers for quite some time, a new library call...
New
felix-starman
I’ve noticed that often when I find myself needing to manually set content-disposition - [MDN] the advice is usually to use URI.encode_ww...
New
pepicrft
Hey folks :wave: I implemented a library that uses macros extensively, and I’d appreciate feedback from people that are more versed at w...
New
weakwire
Hello people, Big fan of elixir & phoenix LiveView. While designing LiveView applications I use LiveComponent extensively. Issue-I...
New
bortzmeyer
Not a replacement for serious authoritative DNS servers, of course, but useful for some uses (such as returning the IP address of the DNS...
New

Other popular topics Top

axelson
This post is a wiki (feel free to hit the edit button near the bottom right of this post to add your own changes!) This post collects co...
239 48342 226
New
chrismccord
Phoenix 1.4.0 released Phoenix 1.4 is out! This release ships with exciting new features, most notably with HTTP2 support, improved deve...
688 31013 112
New
msaraiva
Surface is an experimental library built on top of Phoenix LiveView and its new LiveComponent API that aims to provide a more declarative...
564 43757 214
New
chrismccord
This release brings a number of exciting features, including integration with the new Phoenix LiveDashboard and Phoenix LiveView. There h...
New
josevalim
Hi everyone, One of the features added to Elixir early on to help integration with Erlang code was the idea of overridable function defi...
New
jay1
Why is it that the mnesia database isn’t the most preferred database for use in Elixir/Phoenix?
New
joeerl
Hello again - after a longish gap I’ve decided I really must dig into Elixir and see what’s been happening here - so I have a few questio...
New
shijith.k
I am trying to start a new phoenix project with elixir 1.9, but mix phx.new does not work. It says that ** (Mix) The task "phx.new" could...
New
AstonJ
We’ve put together this wiki for Phoenix LiveView - please feel free to add any info you feel is worth including. What is Phoenix LiveV...
New
PeterCarter
There are pre-rolled solutions for other frameworks that do work. However, Phoenix does not seem to have these. Have people had good expe...
New

We're in Beta

About us Mission Statement