ycastor

ycastor

Commanded with Extreme EventStore: How to set MaxSubscribers

Hello! I’m building an application using Commanded with the Extreme EventStore Adapter and Ecto Projections and i’m having a difficulty to run it in a clustered manner.

By default, when i create a projection the Persistent Subscription on EventStore is created with the limit of 1 subscriber per subscription, in that way, after the first node connection to the eventstore, every other node will throw a subscription_already_exists error.

I took a look on the documentation for extreme and extreme adapter and i have not found a way to increase this number, i took a look at the code and found that the SubscriptionsSupervisor for the Extreme adapter has the option Keyword.get(opts, :subscriber_max_count), but i have not found any way to set this value from the Projection or through the config file.

Does anyone know how can i increase this number?

Thanks in advance!

Marked As Solved

slashdotdash

slashdotdash

I’ve added an issue to the GitHub repo for the Commanded extreme adapter to allow configuration of the subscriber max count for persistent subscriptions.

The limit is set to 1 by default to ensure that only one event handler can be subscribed at a time. This is because event handlers usually want to process events in order. As an example, Commanded Ecto projections uses the event number for idempotency checking to ensure that events received more than once, due to at-least-once-message delivery, will only be projected once. If you were to have the same projection process running on multiple nodes subscribed to the same persistent subscription then events would be processed concurrently and likely out of order. The idempotency check would mean that any out of order events would be skipped because they would be assumed to have already been processed. The current approach is to have a single connected subscriber which guarantees event ordering.

In a multi-node deployment with distributed Erlang the :global module is used to guarantee that only one event handler process will be running within the cluster. This ensures only a single persistent subscription is connected and guarantees event ordering. If you don’t use distributed Erlang then you will have an event handler process started on each node. Unfortunately the extreme library doesn’t handle the case where connecting to a persistent subscription fails due to too many subscribers. This is a limitation that needs to be addressed. It could implement some retry mechanism when connecting to the subscription fails which would allow multiple handlers to be started but only one would connect and actually process the events. This is how the Elixir EventStore using Postgres is implemented.

Where Next?

Popular in Questions Top

9mm
I am constructing a JSON object (map) and I need to conditionally set a field. I’m trying to write proper elixir-way code… and I’m at a l...
New
mcarvalho
What is the difference between System.get_env and Application.get_env? For example, what are best practices to use one versus another.
New
greenz1
I have a phoenix application from which a user can download multiple(5-6) files of size 1MB. I couldn’t find anything related to sending ...
New
stefanchrobot
What’s the safe way to decode a JSON string into a struct? I want to avoid calling String.to_atom. Jason.decode can give me a map with st...
New
fayddelight
I tried installing elixir 1.11.2 erlang 23.3.4 via asdf in my zsh shell. Enabled the versions locally and globally. When I list them ...
New
Qqwy
Original source of discussion: This topic on the Pragmatic Programmers’ Functional Web Development with Elixir, OTP, and Phoenix forum. ...
New
bsollish-terakeet
Credo is smart enough to check for (something like) this: assert length(the_list) == 0 with this response: Checking if an enum is empt...
New
komlanvi
Hi everyone, I was playing with phoenix liveView but I run into an issue. I have a form and want to validate each input text when the te...
New
dotdotdotPaul
Okay, I’m having a heck of a time trying to figure out how to best handle the validation of belongs_to associations in Ecto. I’m sure I’...
New
openscript
Hello! Sorry for this astonishing simple question, but I’m really stuck. I try to set up the intellij-elixir plugin, but I don’t know ho...
New

Other popular topics Top

danschultzer
None of the current solutions worked well for me, so I went ahead and built a user management system from scratch. This project took far...
548 29377 241
New
vertexbuffer
Hello, can anybody help here..? I have a list of players and I what to delete an element, but every for loop the list is reverting to ori...
New
senggen
Erlang/OTP 25 [erts-13.2.2] [source] [64-bit] [smp:8:8] [ds:8:8:10] [async-threads:1] 15:22:35.803 [error] gen_event {lager_file_backend...
New
chrismccord
As promised, the first release candidate of Phoenix 1.3.0 is out! This release focuses on code generators with improved project structure...
New
shahryarjb
Hello, I have map which I want to convert it to string like this: the map: %{last_name: "tavakkoli", name: "shahryar"} the string I ne...
New
msaraiva
Surface is an experimental library built on top of Phoenix LiveView and its new LiveComponent API that aims to provide a more declarative...
564 43622 214
New
JakeBecker
TL;DR: I’ve just released an implementation of Microsoft’s IDE-independent Language Server Protocol for Elixir. It adds language support ...
1144 53690 245
New
AngeloChecked
What learn first? Rust or Elixir Hi Elixir community! I’m here because i want learn a new language. I’m a junior developer and mainly i ...
New
dokuzbir
I want to highlight html closing tags when i click a html tag. That works in .html files but doesnt work for html.eex templates. How can...
New
AstonJ
Seen any cool LiveView demos, sample apps or examples? Please post them here! :003:
New

We're in Beta

About us Mission Statement