Wiring up a Dashboard for Dynamic Supervised GenServers

Paperelectron

Hey all, I’m looking to get some general advice on how to wire up a bunch of Dynamic Supervised GenServers, named via the registry, to a LiveView frontend. I’d like them to be able to broadcast updates about what they are doing, and have the individual list members update.

I’m currently using Registry.select to get the list of servers, but I’m unsure about how to go about handling adding and removing new named Servers as they come up. I’d rather not put any of the data about these servers in the DB, as they are meant to be pretty ephemeral.

My gut feeling is I should have a separate GenServer that handles Subscriptions and the overarching state of the list that I can then use LiveView to call into/receive messages from. Would I use this GenServer to handle the creation/deletion of the child Servers? Use Monitors? This feels like I’m stepping on the toes of the Registry a bit? What happens when I want to cluster this?

Basically, this is the type of problem I want to solve generally using Elixir, and I don’t yet have enough experience to know the correct way to approach it.

Not looking for any detailed code just a bit of a discussion on how those with more experience would architect something like this.

7 comments

#architecture

2 526 7

2024-01-19 12:20:00 UTC

Showing Posts 1 to 7

Show Best Posts
Show All Posts (oldest first)
Show All Posts (newest first)

al2o3cr

A fairly common pattern would combine an initial call to Registry with PubSub:

the LiveView fetches the “current list” from the registry
it also subscribes to a “new server started” pubsub topic (and probably a “server ended”?)
when a server starts, it sends a pubsub notification of that

Adding things like clustering to the mix makes handling cases like “the whole NODE these servers were on went away”; you may need to include some kind of liveness-checking in your LiveView.

Post #1

Paperelectron

So that pattern was my first port of call, then I got to thinking about how I would filter the list if I had 1000 of them and needed to filter and paginate. I started to think about basically doing this in reverse, ie: Broadcasting into the running GenServers with a filter and having them respond if they matched then using those responses to build the list.

Then this leads me into wondering how I would then update individual items on the list as their underlying state changed, haha, its madness.

I figure clustering anything raises the complexity level at least an order of magnitude, even for things designed to be clustered.

Post #2

schneebyte

pg — OTP 29.0.2 (kernel 11.0.2) might be able to handle that out of the box.

You can monitor scopes/groups and get notifications when they join/leave (spawn/die).
Group names can be any Erlang term, so with the right scope/group structure it can be very flexible.

Maybe the GenServers could push state updates instead of polling them for an update.
Have the liveview process register to one or multiple groups {:i_want_notifications, :topic1} and the individual GenServers can send state updates to all members of that group.

Post #3

dimitarvp

Good recommendation though I’d say we should advocate for pg2 instead.

Post #4

LostKobrakai

pg2 is (unintuitively) deprecated since OTP 23 and gone in OTP 24 in favor of then newer pg.

Post #5

dimitarvp

Me not paying enough attention strikes again. [sighs]

Thank you for the correction.

Post #6

Paperelectron

This is good, pg seems to be designed to handle exactly this kind of scenario. I’ve known how to do this using an external data store, as I’ve implemented it in other languages, but it was/is important to me to do this all inside of BEAM, reaching for outside tools felt like the wrong way to handle it.

Post #7

— All posts loaded —