IMO worrying about this is premature; there are a lot of other problems you’ll need to solve before this particular one needs optimization.

What you’re describing with ack and finished messages sounds a lot like two-phase commit; there’s a substantial literature describing different approaches to solving that coordination problem.

The theme of your game has me wondering: should these interactions be instantaneous? The speed of light is already an issue in modern distributed systems, and interstellar distances (even with silly-fast FTL communication) would make it even more so.

Another thought: some real-world transactions are coordinated through escrow services. What about setting up something like that? A possible process:

• party A wants to buy 1000 widgets from party B for a total of $10000
• party A transfers the $10000 to the escrow service
• party B transfers the 1000 widgets to the escrow service
• the escrow service releases the money and the widgets

In a system context like this, you’d typically have the “transfers” above include a timeout so that if the escrow doesn’t complete the resources eventually revert to their original owners.

One note: the name escrow service shouldn’t be read as implying there’s one process doing this. It would seem simpler to spin up a new process per escrow interaction, to keep the state machine straightforward.

In Elixir you can have thousands and even millions of processes in a machine, BEAM is highly optimized and the amount of resources they cost is minimal.

As for state taking up memory, why would this be an issue? Your application needs state somehow, you will have to save it somewhere.

Ahh, cunning solution. I like it

So, from what I can understand, your problem is that when you make a transaction, between several nodes, that transaction needs to be atomic in the sense that no other operations can take place while the transaction is happening so you can avoid inconsistent states.

This is a tough problem, but one with many solutions. One of way avoiding state on workers would be to have your state on an ETS table, and then have a mediator that changes and updates state for all processes. This mediator would eventually become a bottleneck but it would also serve as a point of synchronization in your system. (Regarding bottlenecks, always benchmark first! A mediator will only be a bottleneck if it handles more messages than it can get!)

Another possible solution would be the use of optimistic locks. Locking in DBs is a world on it’s own, but I find that optimistic locks, besides having great performance, do work in a lot of cases when you have few collisions:

Aside from that, the only remaining solution I can think of is defining a worker communication protocol, which you already have defined and explained in your post.

Anyway, hope it helps somehow!

Hi,

In Elixir you can have thousands and even millions of processes […] Your application needs state somehow, you will have to save it somewhere.

Thank you for your answer. I know that I can have more processes than I would ever need, but yes the main problem with that is memory. It is a game so state can be pretty complex, and I do not want to have thousands of starbases state idling for nothing. With this architecture I can keep a low memory profile. Actually I could have processes that hold state for faster read/writes (instead of database/disk) and shutdown with a timeout ; wether the state is written to a transient GenServer with timeout / an EST table or directly do disk (I’m going to start with that) is another question I believe, the main problem beeing queuing updates and multi-state transactions.

Another thing with processes is errors handling. If I want to act on two states and rollback on errors, my state-holding-update-handling processes will need a way to cancel the last update, or at least a savepoint/commint mechanism. Whereas handling the two states update in a single process and rolling back is just a matter of not saving the new states.

So, from what I can understand, your problem is that when you make a transaction, between several nodes, that transaction needs to be atomic in the sense that no other operations can take place while the transaction is happening so you can avoid inconsistent states.

Not nodes, just states. Everything that I describe will happen on a single node. But yes, it is a transaction/concurrency problem. A mediator would be kind of equal to have only one worker. It is simpler, that is true, but I like the idea of having concurrency there. Maybe that is my mistake.

Optimistic locks are interesting and may be applicable in my case, I will research on that point.

Thank you very much.