Hi all, I’m having trouble on the final mile of solving a problem using Mnesia. The use cases details are:
- I have multiple nodes that each track some local in-memory state (for me these are specifically streams, so I will call them that)
- Each node needs to be able to ask a small set of questions about the global state, specifically “what are all the streams that are running?
- The cluster is deployed on fly.io, which auto start/stops machines based on usage.
To accomplish this, Mnesia seemed like a natural choice, being built-in to the BEAM. However, I keep running into the same problem which manifests in a few ways: Mnesia wants to know on start-up which nodes to put a table on.
- the first issue with this is dns_cluster, which fly.io uses by default, discovers nodes after the application is started. I’d think this is fine, because my schema is more or less “stream ID => stream metadata”, and stream IDs are unique (and if there is a conflict, some kind of handling can be defined), so this connect-after-start could simply merge Mnesia tables. This also is the same problem as scaling to another region without wanting to re-deploy the application.
- The next problem is Mnesia tables don’t like to merge. If they were defined with different node lists, one of them essentially has to be deleted while the other is replicated to the new node. I feel like I could program this but it just sounds like a pain to manage, feels like a solution should already exist, and feels like it means a network split is impossible to smoothly recover from. Even if one of the tables is empty, as long as it exists, Mnesia makes a fuss.
So I am wondering:
- Is there a better, simpler solution for my use case that I’m not thinking of?
- If Mnesia can actually work well here, what am I missing about table merging to make it possible?
Thanks for any insight!
