beamologist

beamologist

ExESDB - a BEAM-Native Event Store

Event Sourcing with CQRS is a technique for building applications that are based on an immutable log of events, which makes it ideal for building concurrent, distributed systems.

Though it is gaining popularity, the number of options for storing these events is limited and require specialized services like Kurrent (aka Greg’s EventStore) or AxonIQ.
One of the strong-points of the BEAM is, that it comes ‘batteries included’: there are BEAM-native libraries for many common tasks, like: storage, pub/sub, caching, logging, telemetry, etc.

ExESDB is an attempt to create a BEAM-native Event Store, building further upon the Khepri library, which in turn builds upon the Ra library.

On the roadmap:

  • integration with pg2 and Phoenix.PubSub for side effects (i.e. read model projections)
  • interfacing with the Commanded library
  • leverage Partisan for clustering

Check it out on Hex or GitHub:

https://github.com/beam-campus/ex-esdb

First Post!

Eiji

Eiji

Interesting idea. I’m curious about plans for the commanded support.

So you plan to create an implementation for the Commanded.EventStore.Adapter behaviour, right? In that case it would be interesting to hear about a key differences between your library and PostgreSQL-based Elixir EventStore / EventStoreDB.

I guess it would be something like In memory event store, but for production. That should give us less required environment dependencies which is great. Are there any other pros for that like speed or security?

What kind of serializer would be needed? Would it work with the Elixir structs as-is or we would still need to use a JSON serializer?

Suppose some project already uses other adapter. Would that require some extra steps for migrating data?

Most Liked

beamologist

beamologist

ExESDB v0.1.0 available!
I am proud to announce that the first useful release of ExESDB is now available as v0.1.0! This release comes with an adapter for the fantastic Commanded Library AND a Phoenix LiveView demo app. Feel free to check it out

On Hex:

On Github

DEMO Video
Comments and Feedback most welcome!

10
Post #7
beamologist

beamologist

Hi Eiji,

It has taken me a few months to finish things up (but then, what is ‘finished’, right?) but meanwhile, the distributed store is there, a HA proxy is there and…the Commanded adapter is there, too!
It is said that the proof of the pudding is in the eating, so the adapter is being developed and tested in conjunction with a working Phoenix LiveView application: an event-sourced demo app that mimics a dashboard for regulating greenhouses. All data (events) is stored in ExESDB, while Cachex is used for read models, thus creating a true BEAM-only application without the need of any external services like EventStoreDB or PostgreSQL.

I must confess, it feels a little liberating, being able to just spin up a few containers and not having to worry about connectivity with other backend components.

Also, since we remain in the same ecosystem, there is no need to worry about (or waste processing power with) serialization. Serialization will only become a topic if and when we decide to create API’s for clients in other languages..

beamologist

beamologist

Hi Tomasz,
Thank you for your feedback.
Next to the advantages you already mentioned (speed, security, no need for extra serialization - events are indeed stored as Erlang terms), I’d like to add the capability to deploy event sourced services as a self-contained BEAM-native release to the edge. This would allow us to leverage for instance The Nerves Project for deployment.
I am also looking in to https://bondy.io for scenarios where you could have 10E+N nodes in the network.
Much of my past work revolved around decentralized and autonomous systems (think Parking Facilities, Vehicles, Agricultural automation, Logistics etc), often in a “spotty” environment, where nodes aren’t always connected. Such systems benefit little from SaaS solutions, if the network is not available. That space could be considered my main motivator to build a BEAM-native event store: as few dependencies on 3rd party services as possible.

The reason for implementing the Commanded Adapter is simple: it is the de-facto event sourcing standard for the BEAM and is as far as I am concerned, feature complete.

When it comes to migrating data from existing stores, I’d argue that’s quite easy, barely an inconvenience: replay the old store and project into the new.
So, indeed: highest points on the agenda are:

1, have Kherpi triggers throw seen events on pg2 (for projections etc…)
2. Dynamic Clustering via Partisan
3. Commanded Adapter
4. Monitoring/Telemetry

Last Post!

beamologist

beamologist

Absolutely! It is still very young, so better let it stabilize a little :slight_smile:
In any case, thanks for the tryouts, feedback and contributions!

Where Next?

Popular in Announcing Top

OvermindDL1
Been making an MLElixir thing (not released yet…) for fun in spare time in the past day. I’m just trying to see how much I can get an ML...
132 14410 106
New
kelvinst
Hey everyone! Well, we made this lib a while ago and now we decided to finally go out and public with it! It’s a tool for creating and m...
New
josevalim
Yes, yet another parser combinator library! Most of the parser combinators in the ecosystem are either compile-time, often using AST tra...
159 19951 141
New
mindok
What is ContEx? A pure Elixir server-side data plotting/charting library outputting SVG. It has nice barcharts in particular and works g...
New
maltoe
Hello! Came here to announce ChromicPDF, a pet project PDF generator I’ve been working on for the past few months. Why another PDF gener...
New
oltarasenko
Dear Elixir community, After a year of development, bug fixes, and improvements, we are proudly ready to share the release of Crawly 0.1...
New
martinthenth
Hello everybody :wave: Recently, some of my colleagues talked about database ids and uuids and their problems, and I remembered the pain...
New

Other popular topics Top

nobody
Hi! In PHP: $_SERVER[‘SERVER_ADDR’] - in Elixir? Searched the docs for ip address and the web, no good results. Thanks!
New
chrismccord
Phoenix 1.4.0 released Phoenix 1.4 is out! This release ships with exciting new features, most notably with HTTP2 support, improved deve...
688 31586 112
New
AngeloChecked
What learn first? Rust or Elixir Hi Elixir community! I’m here because i want learn a new language. I’m a junior developer and mainly i ...
New
Patoshizzle
After calling mix ecto.create I get this error: 17:00:32.162 [error] GenServer #PID<0.412.0> terminating ** (Postgrex.Error) FATAL...
New
Harrisonl
We have an ECS cluster with 4 services, where each task joins a single cluster, via discovery ECS discovery service. Currently when I de...
New
AstonJ
Posting this to see if we can make things easier for people to get into Neovim. If you use Neovim and have a favourite distro please let ...
New

We're in Beta

About us Mission Statement