Are you sure the supervisor of the registry didn’t crash, bringing down the registry together with it?
If you don’t trust Registry, you probably shouldn’t trust DynamicSupervisor either, they’re written by the same folks and do similar work monitoring and tracking processes.
I can’t tell what happened exactly.
The setup in that project is like this:
Supervisor:
-- Registry
-- DynamicSupervisor
------ DynamicSupervisor
---------- GenServer
---------- GenServer
---------- GenServer
------ DynamicSupervisor
---------- GenServer
---------- GenServer
---------- GenServer
...
I had cases where the Supervisor supervising the GenServers died and restarted but I cannot remember that the top level supervisor ever died.
What was your restart policy for the Supervisor? If it was one_for_all than that’s exactly the behaviour you would be seeing. Everything your DynamicSupervisor would crash, everything under the Supervisor would be restarted.
So, I played around a bit and created an example add.
I used a single Registry which seems to work fine. Code can be found here:
https://github.com/PhillippOhlandt/dynamic_supervision_communication
For now, I just added the mechanism to start a new unit which then starts a service and two worker processes. Gonna add unit listing/deletion/state retrieval too. In the mean time, feel free to look over the code and suggest things that could be improved (concept wise).
Here is an example console output:
iex(82)> MyApp.add_unit "Unit 1"
{:ok, #PID<0.1344.0>}
iex(83)> Unit 1/:worker_two got work to do
Unit 1: Message from :worker_two: I don't really wanna do the work today!
Unit 1/:worker_two got work to do
Unit 1: Message from :worker_two: I don't really wanna do the work today!
Unit 1/:worker_one got work to do
Unit 1: Message from :worker_one: I don't really wanna do the work today!
Unit 1/:worker_one got work to do
Unit 1: Message from :worker_one: I don't really wanna do the work today!
nil
iex(84)> MyApp.add_unit "Unit 2"
{:ok, #PID<0.1351.0>}
iex(85)> Unit 1/:worker_one got work to do
Unit 1: Message from :worker_one: I don't really wanna do the work today!
Unit 2/:worker_one got work to do
Unit 2: Message from :worker_one: I don't really wanna do the work today!
Unit 1/:worker_one got work to do
Unit 1: Message from :worker_one: I don't really wanna do the work today!
Unit 2/:worker_two got work to do
Unit 2: Message from :worker_two: I don't really wanna do the work today!
I usually name my GenServer like {:global, {:whatever_name, unique_id}} , then at runtime, look up pid with GenServer.whereis/1, do that every time you need to send a message, just in case the process was restarted in between. If only one instance will be running, use __MODULE__ works too and saves the lookup.
I doubt you need Registry for this.
Got another feature I would like to have.
In my setup, given I have started some units. Now I want to “label” them.
Lets say I have Unit 1, 2 and 3. Unit 1 and 2 get the label/value “foo/bar” and “foo/baz” and Unit 3 gets only “bla/blub”.
Now I want to reach out to all Units with the label/value “foo/bar” which in this case are Unit 1 and 2.
There are two ways of doing this which have their own tradeoffs:
Using another Registry with duplicated keys. When I send a unit to register under a certain label/value, it registers itself in that new Registry. The good thing is, if that unit dies, the Registry will remove the entries. The bad thing is, if the unit dies and restarts, the label information are lost. I would need to store them somewhere else to recover them.
Using a simple GenServer to store a map of label/value entries and their connected “unit names” (not pids). The good thing is, if a unit gets restarted, I don’t lose the label information. The bad thing is, the reference map could get outdated and inconsistent.
Which way would you prefer?
I think you are looking for GitHub - uwiger/gproc: Extended process registry for Erlang · GitHub , one of its feature is “Register a process under several aliases”
When you start GenServer with {:global, term()} you can only have one instance per cluster for a specific term(). Starting with {atom(), node()} would have one per node. Different runtime characteristics for different use case.
With {:global, term()} you could use GenServer:call/cast( {:global, term()}, request ) instead of doing a whereis/1 each time. GenServer would route the request to the correct node for you.