I am currently working on an application composed of several micro services. Having never done an Elixir project in a professional way, I don’t know how to organize my code properly and if there are conventions to follow.
Here are some details about the current project (or as I see it):
- I need a database (PostgreSQL)
- I use a caching system for the shared state between my services (for my tests I use Redis but I will migrate to
- I currently have about ten (micro)services
How is the application currently organized?
- It’s an umbrella project
- There is an application managing the entire database (which uses
- There is an application called
core that will contain the code shared by several services (data serialization for Redis for example)
- There is one application per service
- Each service has an
Interfaces module that exports functions to communicate with it (GenServer call/cast or rpc)
When a service starts, it will look for everything it needs in the database and cache it for later access.
I’m trying to find out if the architecture of my code is correct.
One of the problems I see for example (I don’t know if it’ s normal) is that all services have several other services as dependencies in
mix.exs (to be able to use
I would also like to know if you know of any “big” open source projects that use more or less the same stack as me (several micro services + a database + a caching system) so that I can study the code.
Thank you in advance.
Hi there, unfortunately I cannot point to any open source project that uses microservices (usually services are very business-specific and thus makes little sense to open source them).
But I’d urge you to take another look at your design on the basis that these services share a common cache and access the same data, which makes me think that the domain modeling and separation of concerns has room from improvement.
Disclaimer: I know absolutely nothing about your business case and your current approach may be totally legit- I’m only speaking from personal experience here
First of all, thank you for your answer.
To give more detail on what I’m currently doing, I’m working on the server side part of a MMORPG.
To understand why I need shared data, here is an example: let’s imagine that I have a service that manages the movement of players, a service that manages chat messages and a service that manages transactions between players for example.
All these services will need to know the position of a player (current map).
- The one who manages movements so that it can update it
- The one who manages the chat because if I send a general message I want to send it only to those on the same map as me and not to everyone
- And the transaction service because I want to check that players are on the same map before starting the transaction.
My first approach was to store the player’s position in the state of a dedicated service but in the long term, it might cause bottleneck problems.
I don’t know any other methods, that’s why I opted for a shared state. If you know one that could work for this case, don’t hesitate ^^
Ok, I think I understand more about your problem now. How about this:
It seems that all activities are centered around players, and players do not share state between them (eg each player has their own inventory, their own position on the map, their own transaction history etc.)
So perhaps it makes sense to model each player as a separate process that shares no state with other player processes.
But then you need to possibly be able to answer questions about location, like who is on the same map as me? Are we close to one another? This concern (map state) I might also model as a separate process holding the current state of the map, but I’d also definitely register for each player the current map (eg by using the Registry module or it’s distributed counterparts) so that its easy to broadcast to all players on the same map, or be able to answer player count on the map etc.
There is also a great opensource game project (ExVenture) that has perhaps concerns similar to your own and couls perhaps borrow some ideas from.
Also you might find this post on some aspects of discord design of value since they also have players logged in many channels simultaneously
Does this caching method have an advantage over the one I currently use?
I mean, in both cases it’s a way to do state sharing, right?
And thank you for links, I’m going to study all of this.
Sure. What I try to do when thinking about possible designs is to minimize or completely forgo shared state. That sometimes may lead to bigger services than others but the benefits usually outweigh the cons.
One possible way to achieve this is by modeling each player as a separate process. Players can naturally interact via messages when there’s a need to (like for example when trading or fighting) but otherwise they are completely autonomous from one another.
Using this approach you would not need a shared cache- each player process would be responsible for managing their own state. All you need then is a way to identify individual players which you can do any number of ways (gproc, Registry etc.)
Then there are some elements in your game like the map on which any number of players can simultaneously be on. You could ‘group’ player processes on the same map so that you have an easy way to address everyone on the map for example without sharing anything else- loose coupling is usually good design (gproc and Registry can help here as well)
In any case as I said above please take all of my advice with a grain of salt. I certainly do not know anything about the game you’re building and haven’t spent nearly as much time as you thinking about it. Use my advice as generic advice if you will, but decide on your own the best design for what you’re building- you’re arguably the most qualified person for this.
Okay, now I understand better what you said earlier.
Thanks for all these tips, I’ll try to make a PoC to see advantages/disadvantages in terms of code, performances and ease of implementation.
How are the individual “microservices” implemented? If you’re using named GenServers, be aware that those are single-threaded - so (referencing one of your later examples) the “chat service” would process incoming call/casts one at a time.
Sometimes this property is a feature (for instance, if the GenServer is mediating access to something that specifically needs one-at-a-time semantics), but it can be a nasty surprise for people used to creating services that run in threaded application servers like many web frameworks.
I tried to design each service so that it could be replicated very easily (another reason why I used Redis).
I have carefully considered the fact that each GenServer can only process one message at a time.
Let’s take the case of chat service for example. As soon as I receive a message from the player telling me that he wants to send a message on the general channel, if I have 3 chat services, I will just take one randomly (I may implement another selection method later), and I will send him my message. This service will then retrieve the different information it needs in cache (here all the players on the map) and send a message to all these players.
Now, let’s consider a slightly more delicate case: the transaction system between players. Here, in order to avoid any race condition or potential duplication bug, I have bet on the fact that each GenServer can only process one request at a time. It’s therefore always the same process that will process the requests of the same player. To do this, I use the
erlang.phash2/2 function on the player’s PID (instead of a random function) to always select the same service to process the request.