Use case for Elixir/Phoenix as a game server - thoughts appreciated!

Questions / Help
, ,
1

Hi there,

Quick background
Long time Ruby/Rails developer who has been working on a strategy multiplayer game in Unity.

I’ve got a use case here using a very simplified game idea and wanted to see if Elixir/Phoenix would work for some or all of it. I broke it out into sections to hopefully allow me to articulate the various use cases in a meaningful and objective way.

Use Case
Imagine a game that is Chess but massive-multiplayer - that is to say, you could have 1000 players playing on the same chess board against each other. Each player has their own set of pieces (rook, king, queen, etc). Each player would start somewhere on a large-enough board (so imagine a 1000x1000 grid).

Ignoring the obvious difficulties with modifying the ubiquitous chess rules to support this, the only mechanic you need to understand is this:

  • all 1000 players will submit a move within 30 seconds to the game server (aka a round)
  • at the 30 second mark, the game server will process every submitted move and calculate the result and update the state of the game
  • the game server will send the result back to all clients and the clients will be responsible for the ensuing animations and so forth

As an aside - the mechanic above is often referred to as “simultaneous turn” which is in the middle of the spectrum between turn-based and real-time. I could have used tic-tac-toe, connect 4 or any number of games out there as an example, so please don’t get caught up on this.

Section 1: Multiplayer usual suspects - login, chat, lobby, etc.
I don’t think this will be contentious in any way - but user authentication, chatting, lobby, game finding shouldn’t pose any issue for Phoenix/Elixir.

Section 2: Client/Server communication and scale
The client would be Unity (C#) and it could simply use https and json to submit their moves to the server. Or web sockets or anything else. The move could be as simple as “Player #234, move Knight to E23”.

Note that the move will be persisted into the database to be processed later (Section 3 below).

I don’t think there would be any issue with using Phoenix/Elixir, even if you had to scale to 10,000 players or 100,000 players.

Section 3: Game server - processing user moves and calculating the result
This is really the crux of my question. Things to note:

  • The state of the chess board would be stored in the database.
  • As users submit their moves, they would get persisted to the database.
  • When the 30 seconds are up, the server will process all available moves (in the database) against the state of the chess board and generate a result set.
  • The state of the chess board will be updated in the database.
  • The result set will be sent back to all clients.

This is where the rules of the game need to be codified in Elixir using functional programming rather than traditional OOP.

The calculation of the outcome for the round in this game would probably be CPU and definitely memory intensive. Not only would you need to store the state of the chess board in memory, you would also need to access the data objects of all the units on this board as they impact the calculations.

Section 4: Game server - AI implementation
There are two cases here:

  1. for players who do not successful submit a move in the 30 seconds then a move will be generated for you.
  2. computer players can also be one of the 1000 players playing

Again, this would be CPU and memory intensive. Looking at traditional AI, you would have in-memory representations of the game state to help calculate the best move based on some sort of score/criteria.

Section 5: Game server - Performance and Scaling for multiple concurrent games
I’m guessing but perhaps the best approach here to support 3 or 4 concurrent games with 1000 players each would be to have a “instanced server” for each.

When a game starts, you would fire up a new server instance with its own micro-database to store the state of the game as it plays out. Clients would access this server directly.

Is this possible with phoenix? I guess it might not be a phoenix question but more of a dev-ops/docker/container question.

And that’s it… if you made it this far, thanks for being patient!

Thanks!

5 Likes
2

Hello and welcome,

Just some consideration, not a global answer :slight_smile:

section 1… as mentionned, this part is not an issue

section 2… You might need a c# websocket client, there are some, but You need to check if they are compatible with latest Phoenix version

I would not use database for direct play, but to persists state, in case of reload

section 3… as previously mentionned, I would not use database as source of truth. There are many of ways to store state (process, ETS) that are much faster than DB.

They are some functional chess libraries, I wrote one available in hex.pm (sorry, it’s not meant to be self publicity)

section 4… We are waiting for the next big thing in Elixir, it’s not yet announced, but it might be AI related

section 5… You should read about OTP. It’s THE library You need to know for concurrent programming.
Don’t forget “Phoenix is not your application”, You can start by building the functional core with Elixir/OTP, then wrap Phoenix as an interface only.

If You don’t know about OTP, You can start with The OTP design principles.

I would use Elixir any time to do such work :slight_smile:

3 Likes
3

this seems like a good use case for phoenix!

4

Seconding Koko’s answers and recs!

My team and I have recently started on a similar path (Phoenix for client auth, channels for client communication) for an app with game state. I started this repo of Elixir Gaming examples + resources to aggregate some of the libs/games/talks/repos we were coming across

There are a couple of links to C# libs and some example networked games that might be helpful; if you have any resources / links / etc to add, please share as well :stuck_out_tongue:

2 Likes
5

Hey, some thoughts (because I have been for quite some time now working on a game using elixir in the backend),

As kokolegorille mentioned you probably do not want to use the database other than for storing a “backup” from which you can reconstruct or replay the state. I would store the intermediate state (when playing) in something as a OTP process (gen_server or gen_statem, either a map or an ETS table owned by the process) that is responsible for each “game/table” and then actually only persist the calculated result of each round to the db. This means then that I could “replay” the game up until whatever point by just applying the calculated results of each round to the initial state (maybe you would also want to store each players “move” per round, but that would be just for better description of the round and not essential to rebuild the state).

The process itself would also be the only way to issue commands and etc, effectively working as a serialization point, it would be uniquely identifiable and only a single instance of it per game would ever be running. This gives you a way of addressing the game for the players if they connect or disconnect (connection error or wtv) as long as you have the ref to the game. It also means that if you can “load-balance” the creation of games/tables across your servers, and they’re connected through distributed erlang, the players can connect to any server and communicate to their game process from any one which should give you the tools after benchmarking to know how many servers you need or the size of them, etc. 1000 players seems ok to be handled by a single process specially because it’s 30 secs windows, probably the last few seconds will see spikes and might need special consideration.

You probably need to do some benchmarking, but either with a Map in the process state or an ETS table, I would store the matrix of the table (1000x1000, it’s 1M entries), where the key would be the cell coordinate and the content a tuple of {player_identifier, piece_identifier}, which should be “relatively” cheap on the memory side and if using ETS constant time access to each key. You can “see” the costs of this easily (with observer or standard lib tooling) and they should be linear.

Because you want to talk between client && server either use websockets or if you can just connect from the client to a regular tcp socket on the elixir/erlang side that’s also a possibility.

To time the users moves, if using gen_statems you have an array of timer functionality built in that OTP process that you can use to trigger things, otherwise you can set timers with send_after and such (although this requires a bit more book keeping to keep in synch it is totally doable).

The only two problems I can see:

  • Computer player deciding their move or automatic play when human user doesn’t make a move - a 1M board is a lot of options and this will probably be a problem independent of what you use

  • processing the moves - this one depends on how your engine is built and the rules it has. If it has to consistently reevaluate previous decisions in light of the result of processing new moves then it can be tough (I mean in the same “turn” decision, regularly backtracking), but if it’s more or less ordered and linear then ETS access speed is constant, and getting two cell’s content, deciding what is the result and updating both cells should be fast - but again you’ll need to try it out.

For the “units/pieces” you have, I would store those using persistent_term (given that they won’t change at all). So getting the “unit” information for any calcs should also be pretty fast - the board coordinate from ETS/Map (ETS very fast key access no matter the size) gives you the owner and the unit ref, get the unit stats from persistent_term using the unit ref, ready to calculate. This also means than those units would be stored only once (per server) and so the state of the game that is kept in memory can be much more lean (if you have more than 1 simultaneous tables on a server, it already pays off, if it’s 1 table per server then it would be pointless).

So if you solve the bots/auto movements decision process, and your engine flow is not regression heavy you’re done because for everything else I think elixir is a great fit (also on code organisation, specially the rules engines, usually pattern matching shines there from my experience)

3 Likes
6

You may have already read to spawn or not to spawn from @sasajuric, but if you haven’t it should help you solidify what should be modeled as a process (and what shouldn’t). Each board = 1 process sounds about correct, and this modeling has multiple benefits like others here mentioned.

For a single process to handle (apply on the board) 1000 moves (ie messages) arriving all at the same time in the worst case it shouldn’t prove too difficult. But perhaps depending on game rules there are some clever ways to partition the board and do some parallel processing, e.g. for parts of the board where players’ pieces do not yet overlap.

For AI communicating over ports with python processes would perhaps make sense as there’s a ton of libs ready to use there.

Scaling to multiple games in parallel should not be a problem even with vanilla erlang distribution scaling across 30-40 nodes should be easily feasible.

Hope these help!

1 Like
7

Wow, thanks everyone.

I must confess I haven’t done a lot of reading on elixir/erlang or phoenix so some of the terms described here (OTP Design principles, persistent_term, spawning, etc) are not ones I’m familiar with … yet!

I especially like the idea of ports. Phase 1 is to build the single-player out which means all of the game engine / rule processing / AI code resides in Unity/C# but structured in a way that it’s not utilizing unity libraries. One of my concerns was having to rewrite OOP C# code into Elixir FP - an idea that I wasn’t particularly fond of.

Also something I need to consider is team skill composition. Right now, I’m a one-man team doing everything working towards a proof of concept. If that works out then I would work and hire towards launching the single player and depending on how that goes, follow up with the multiplayer capabilities.

There’s a certain simplicity with having everything (client and server) as the same tech. One thing I will have to vet is Unity’s multiplayer modules that are under development. As a Ruby on Rails dev who’s been around the block, I know how expensive good developers are and I suspect senior Elixir/Phoenix devs will probably be quite rare and expensive.

Does anyone know of any success stores with Elixir being used as a backend server game engine?

1 Like
8

Have you already looked into https://github.com/dotnet/orleans ? Example Halo game services are build with it.

9

Great question and great discussion! I don’t have anything technical to add, but one comment on skills availability:

A top developer would become productive in Elixir quite quickly regardless of background, and you could always get key architectural decisions reviewed / vetted by one of the awesome consulting companies active in this space (which has the added benefit of supporting the tooling) - see https://sponsors.elixirforum.com/ for some options. Furthermore, useful, maintainable functionality per $ spent with an expensive senior elixir developer would, I’m sure, beat almost any other option for this kind of application.

2 Likes
10

Didn’t square enix use elixir as a backend for something?

11

There are DemonWare that use Erlang and Python for some game servers like Call of Duty (https://www.erlang-factory.com/upload/presentations/395/ErlangandFirst-PersonShooters.pdf) and Riot Games which also used Erlang for its messaging service (Riot Messaging Service | Riot Games Technology).

1 Like
12

There are so many “depends” what the game is.

Simple games such as chess shouldn’t be a problem, game session state consists of 64 elements in simplest implementation. There is no frames, and state is changed only on user interaction. The rest are functions that should guard legal/illegal moves and are called only when user interacts with backend.

But games such as Call Of Duty, that should handle N player is rather challenge. Here you need to think in frames, hence, you need to process all relevant data in “single pass”. This is not concurrency problem, it is rather parallelism problem when you have to execute game logic in single frame, in other words you have max 16.7ms to process state and respond to all clients.

I don’t say it is impossible, but it depends how demanding processing is. Erlang gives you process isolation out of the box, and that is fine for communication, but you will find challenging the part with parallel processing.

I do think that ECS can be implemented, where state is stored in ETS in form of components, then use pipeline (GenStage) processes to run game logic you need. For instance here is one frame:

  1. query player bullets, recalculate their new positions store in ETS
  2. query players and bullets, check if player entity is in collision with bullet… store player status and bullet status in ETS
  3. query survived players and move them in requested direction

Of course, you can filter or split entities in stage before the one that will execute above, then process each chunk in parallel. Good example is when some players are in one room while others are in other room. Then you can split processing in two parallel processes and handle each separately, since bullet can’t pass through room walls…

Of course, ECS is not the only game engine interanls, there are others that maybe be better fit for your case, but at the end, you will have to think how to slice data into smaller chunks so you can quickly process it.