Same piece of data in multiple GenServers' state

Hi, this is my first post here :wave: The forum has been a great source of valuable information that I personally appreciate but I never got involved so far. Thanks to the community for the effort :slightly_smiling_face:

I’m just curious about something and I’d like to read your ideas/opinions. Imagine you have the same piece of data stored across different GenServers and it’s only used in a read-only fashion. If it was a large binary, I assume it would be stored in the shared Heap, but if it’s something else how would you optimize for memory usage in case the data becomes sizeable?

Just to give a bit more of context, I’m thiking about maps in multiplayer games. Somehow they store which areas are not allowed for the players to move (rocks, lakes and so on). If the map is big enough it doesn’t seem a good idea to store the map along with each player process.

That’s more or less my question. There are options like ETS tables, just wondering what others can you think of.

I’ve played a bit with the idea of using module attributes as storage on compile time and passing just module names to the processes, then calling something like board.some_fun/x.

Thanks!

3 Likes

:wave:

Maybe persistent_term.

1 Like

Maybe persistent_term.

Didn’t know about this one. It’s optimized for reading data that is infrequently updated so yes, it could be an option.

What I like about storing data in a module is that the work is done during compilation and it doesn’t need anything additional except loading the module in the VM. I think it makes sense for things that hardly change and are known ahead of time.

This might cause problems during development when the module is not recompiled and stale data is used. Although there are work-arounds like @external_resource and Application.compile_env for this.

But yeah, if it works for you, then it’s a good-enough approach.

Be very careful with persistent_term as updating in it can be very costly, worst case a complete gc of the whole system.

2 Likes

It sounds to me like you already have the general idea. You’ll likely want the map stored in state in one process (whether it’s a GenServer, :ets, agent, etc) and anything player specific stored in a process specific to that player.

I’d stay away from persistent term unless you absolutely are sure that the map will never need to be updated.

Thanks for the feedback :+1:

I still think persistent_term is just fine, as long as you can guarantee rare writes. A map in a small game actually seems ideal for this: how often do maps change anyway?