Event sourcing in Elixir - Without CQRS

Questions / Help
1

Greetings,

I’ve been looking around for a good Event-Sourcing sample project in Elixir but all I could find is the great job of Ben Smin (@slashdotdash).
Which is really great but I do not believe in CQRS complexity for a simple project.

Does anyone have any suggestions where I could study more about Event sourcing in elixir without slicing apart my read and write interfaces?

Thanks in advance,
Blokh

2 Likes
2

It’s not really ready yet (a few upcoming changes in the pipeline), but https://github.com/CargoSense/fable is a great intermediate between not doing es and going all in with commanded.

4 Likes
3

Thank you @LostKobrakai!
But I am searching for an ES project.
Just not a CQRS ES project - which divides the read from write interfaces.

4

You might like

I am not sure it is a requirement to have multiple database, isn’t it just a separation of concern between read and write?!

3 Likes
5

Thank you very much! @kokolegorille
I will watch it ASAP, it may provide me some clearance of what I am searching for.
Anyhow.
From my understanding -

Event sourcing is for each action happening in the system - It generates an event regarding this action.
the action is being stored and later on, you can do whatever you want with it. call the aggregator, or later on.
like Registry in elixir only here the data is saved to the event_store.

CQRS took it to another level which is optional. you have one logic for reading the information. and one logic to dispatching the data into the event_store/Database.

I want to keep my read and write logic under the same interface, including the validations, queries and so on.

If I am wrong, could anyone please clarify it for me?

6

Hi there, the problem that cqrs tries to solve is that because of the way data is written in the event store it’s very difficult and/or inefficient to do aggregate queries against that data.

For example lets say your domain is banking accounts, and each event represents a transaction in some customer’s account. Without a separate read model optimized for queries it would be really difficult to get stats like “give the top ten accounts that have made a deposit over x amount last month”

So unless you never read that data outside your main application (eg no admin screens presenting aggregate info over your data), you’re probably going to need a separate read model- and that means buying fully into es+cqrs

1 Like
7

Event Sourcing

  • Every software developer who has used version control has interacted with an Event Sourcing system.
  • If you need to explain Event Sourcing to the business, use accounting ledgers.
2 Likes
8

@bottlenecked @peerreynders hank you very much for taking the time to respond.
@bottlenecked I agree when you see events as “actions”, then you generate the “new state of a different projection” by those actions.
I see events as changes in state of a model.
e.g. for bank - the transactions generate a new event in the balance_store - that includes all your balances from the open_account event.
the balance is transactions history. and the last state of the store is the current state.
anyhow I would like to save states instead of actions.

@peerreynders Thank you, I will check the sources out.
Thank you for reaching to me.

9

I see… so you’re thinking of capturing all current state in what is effectively a data dump. I think I can see a few problems with this approach:

  • space efficiency: you’d need multiple times as much disk space to capture the entire state every time a change happens
  • cpu cost: serializing the entire state to write to a database would be more costly than simply writing the action like ES canon suggests
  • loss of clarity: one of the main benefits of event sourcing (if done right) is that a business user can just take a look at the actions performed in the order they occurred and get a clear picture of what exactly happened (events work as a log). To do that in the system you’re thinking of, one would have to diff between state captures to find out what happened

I hope the above help you :slight_smile:

1 Like
10

To be clear, Fable is ES, not CQRS, which is what I think @LostKobrakai meant to say. You could probably build a CQRS system on top of Fable, but that isn’t how we use it. Here’s a post where I talk about it in more detail: Opinion on file & memory based event sourcing system - #4 by benwilson512

1 Like
11

You can use Commanded for event sourcing without requiring CQRS. Commanded focuses on the write model, but provides the Ecto projections library as one way of projecting events into a SQL database. There’s an undocumented function to access aggregate state (Commanded.Aggregates.Aggregate.aggregate_state/3) which would allow you to use Commanded with a single model for writing and to later query the state of the aggregate.

The reason why this isn’t the preferred approach (IMO) is because although the query model initially closely resembles the write model, soon they will start to differ in structure or access pattern (e.g. reporting, aggregating data and multi-entity views). Hence why I opt to go with separating reads from writes from the outset (CQRS) when using event sourcing.

4 Likes
12

@bottlenecked Thank you very much again.
I agree with all of your points. I will try to redirect my thinking toward the CQRS kind of thinking.

@benwilson512 Thank you very much! I will review the post ASAP.

@slashdotdash Thank you very much! I’ll check out them the aggregate_state/3 method.
Although I think I am leaning toward CQRS currently.

I have a lot to learn tho.

Thank you very much to you all that invested your time to respond to my questions. you are awesome.

1 Like
13

Busting some CQRS myths
https://lostechies.com/jimmybogard/2012/08/22/busting-some-cqrs-myths/

Myth #1 – CQRS = Event Sourcing and vice versa

Event sourcing often fits well with CQRS, as it makes building and updating read stores over time a bit more straightforward in eventually consistent models. Additionally, the aggregates living in the command side for behavior-heavier systems and task-based UIs more obvious.

However, event sourcing is a completely orthogonal concept to CQRS . While they fit well together, doing CQRS does not require event sourcing, and doing event sourcing does not automatically mean we’re doing CQRS.

1 Like
14

Hi, @Blokh,

There is a way to “plug” Event Sourcing easily in combination with Ecto. Here is simplest example

defmodule Sourced.Repo do
  use Ecto.Repo, otp_app: :sourced, adapter: Ecto.Adapters.Postgres
end

defmodule Sourced.DomainEvent do
  use Ecto.Schema
  import Ecto.Changeset

  schema "domain_events" do
    field(:name, :string)
    field(:source, :string)
    field(:data, :string)
  end

  def append(params) do
    fields = [:name, :source, :data]

    __MODULE__
    |> struct([])
    |> cast(params, fields)
    |> validate_required(fields)
  end
end

defmodule Sourced.User do
  use Ecto.Schema
  import Ecto.Changeset
  alias Sourced.DomainEvent

  schema "users" do
    field(:first_name, :string)
    field(:last_name, :string)
    field(:age, :integer)
    field(:email, :string)
  end

  def create(params) do
    required = [:first_name, :last_name, :email]
    fields = [:age | required]
    # this should be regular path, where you cast params to your model
    __MODULE__
    |> struct([])
    |> cast(params, fields)
    |> validate_required(required)
    # at this point we will know if change to model is valid
    |> case do
      %{valid?: false, errors: errors} ->
        # if change didn't pass validation, return error
        {:error, errors}

      %{changes: changes} = model_changeset ->
        # now we want to generate and "pack" events
        # note that we can split `:change` to several events
        # but below I put them all in single event
        event_changeset =
          DomainEvent.append(%{
            name: "user_created",
            source: Atom.to_string(__MODULE__),
            data: Jason.encode!(changes)
          })

        # use ecto multi and explicit repo transaction so we make sure that
        # all data is stored!!! otherwise you will endup with incosistency
        # between state and event journal
        Ecto.Multi.new()
        |> Ecto.Multi.insert("create_user", model_changeset)
        |> Ecto.Multi.insert("user_created", event_changeset)
        |> Sourced.Repo.transaction()
    end
  end
end

And here is test case (again, very simple)

defmodule SourcedTest do
  use ExUnit.Case

  setup do
    # Explicitly get a connection before each test
    :ok = Ecto.Adapters.SQL.Sandbox.checkout(Sourced.Repo)
  end

  test "should persist new user and domain event" do
    assert {:ok, %{"create_user" => _, "user_created" => _}} =
             Sourced.User.create(%{
               first_name: "foo",
               last_name: "bar",
               email: "foo.bar@example.com"
             })
  end
end

One disclaimer for about example. What ever CQRS is trying to solve, it is NOT solved in this example. So either you have to solve it by your self or use 3rd party lib.

15

Event sourcing is more than just writing to a table called events. The “sourcing” bit of event sourcing means that you are driving state exclusively off of the event log. Your example here simply writes an event alongside the normal table insert, it doesn’t drive the table insert off of the event. To do this properly you also need order guarantees: that each event is processed in order to other events.

2 Likes
16

That is true, and that is why I put disclaimer at the bottom and this is just one possible path and not solution

1 Like
17

Thank you @mjaric and @benwilson512.
I’ve took the CQRS approach.
I will take a while to study how to properly code in elixir especially with the CQRS approach. but that’s fine I’ll do my best :slight_smile: