henrysdev

henrysdev

GenServer Memory: Best practices for large internal state accumulation

I have a multiplayer game backend that hosts many simultaneous game rooms as GenServers under a dynamic supervisor. Over the course of a game, one of the responsibilities of a game room GenServer is to track/accumulate the state of the game from beginning to end. After the game ends, the state is written to the database for tracking purposes (recording the result of each round, who won, etc etc).

As I add more and more intricate game state to be tracked, I’m starting to wonder about the scalability of such an approach.

Is there a limit on how large the internal state of a GenServer can be? Is there a good way to benchmark/graph the memory usage for a single GenServer using observer or something else? Will GenServer performance suffer due to having a huge internal state map (assuming immutable copies need to be made)?

Alternatively, I could persist and flush the game state data as I collect it rather than do one big series of DB writes at the end. However, the data model is hierarchical in nature so this would be a bit wonky. I’ve also considered storing the historical game state in an Agent or ETS temporarily before game end - would love any insight/second opinions!

Marked As Solved

APB9785

APB9785

Creator of ECSx

So, you’re using one GenServer per game, which might go from ~4kb to ~400kb over ~5mins, and then at the end of the game, the GenServer saves some of the data to the database, and kills itself? Is that right?

Sounds perfectly reasonable to me.

Also Liked

APB9785

APB9785

Creator of ECSx

I just started up a GenServer to hold a list of Strings, and added ~1mil characters at a time until I got to several hundred MB memory usage. No noticeable slowdown, neither with inserting new data, nor retrieving the data. Obviously this is not a very robust test, but it’s enough for me to think that if there is a performance issue, it would probably be from an inefficient update algorithm, or insufficient hardware, rather than a limitation of GenServer itself.

henrysdev

henrysdev

Ah yes - let me give more context

  • Games are pretty ephemeral; they are tracked entirely in memory during gameplay - they only last like 5 minutes and are not high stakes. I’m not worried about recovering a game in progress if it crashes.
  • A game consists of multiple rounds (between 3-10 rounds)
  • A game room has between 2-10 players in it, but I’m hoping to have many more at some point..
  • Every round, every player in the game submits a json blob payload to the server with a max size of ~4kb. This is where significant mem usage would accumulate over time in storing the history. I ultimately store these blobs in a json column in Postgres.

While I do know that Agents and ETS are also just in-memory storage solutions, I’m wondering if it’d be advantageous to split up the state between multiple processes vs just the one genserver.

Where Next?

Popular in Questions Top

Darmani72
If I have a post route which an argument: post /my_post_route/:my_param1, MyController.my_post_handler How would get the post params ...
New
lastday4you
I wanted to check elixir version in phoenix because i found that my elixir is 1.5 but when i use Enum.chunk_by it said the function is un...
New
chrisalley
ExUnit now has describe blocks which is a welcome addition coming from RSpec. In the docs, it states that nested hierarchies of describe ...
New
gshaw
What is the idiomatic way of matching for not nil in Elixir? E.g., First way: defp halt_if_not_signed_in(conn, signed_in_account) when...
New
fireproofsocks
I’m working on defining a simple Ecto schema for a table (in PostGres), but I don’t see where I can define a column as NOT NULL. Conside...
New
shahryarjb
Hello, I have map which I want to convert it to string like this: the map: %{last_name: "tavakkoli", name: "shahryar"} the string I ne...
New
joeerl
Hello again - after a longish gap I’ve decided I really must dig into Elixir and see what’s been happening here - so I have a few questio...
New
Qqwy
Original source of discussion: This topic on the Pragmatic Programmers’ Functional Web Development with Elixir, OTP, and Phoenix forum. ...
New
openscript
Hello! Sorry for this astonishing simple question, but I’m really stuck. I try to set up the intellij-elixir plugin, but I don’t know ho...
New
jononomo
For some reason my phoenix channels are working for me in my local dev environment, but as soon as I deploy via Docker, I get a 403 error...
New

Other popular topics Top

JeremM34
Hello, how can I check the Phoenix version ? Thanks !
New
electic
Hi, I am new to Elixir. I am trying to use the DateTime component to insert a date into MySQL however the there seems to be no way to fo...
New
fireproofsocks
Forgive me if this is obvious, but how does one delete a database record WITHOUT selecting it first? Ecto.Repo — Ecto v3.14.0 has exampl...
New
pmjoe
I have a relationship of love and hate with Elixir. Lots of things are just absolutely right, but there are some things that are kind of ...
New
boundedvariable
I am going through the kafka architecture. All the features what the kafka is providing are already in Erlang. I would like hear your opi...
New
romenigld
I am trying to run a deploy with docker and I successfully runned with this command: docker build -t romenigld/blog-prod . but when I t...
New
Brian
What is the proper way to load a module from a file in to IEX? In the python world, doing something like this pretty standard: from ....
New
shijith.k
I am trying to start a new phoenix project with elixir 1.9, but mix phx.new does not work. It says that ** (Mix) The task "phx.new" could...
New
hariharasudhan94
I would like to know what is the best IDE for elixir development?
New
vonH
In asking this question I am more interested about the expressiveness of the language itself and less concerned about the availability of...
New

Latest on Elixir Forum

We're in Beta

About us Mission Statement