I’m trying to learn and understand the tradeoffs between two different options for keeping large (constant) terms in memory. By large, I mean probably not more than 25MB.
Taking advantage of the VM feature that if a constant term is returned from a function, it will be placed in a shared heap and thus not copied between processes. (Discord has previously blogged about this, and even created the fastglobal package.)
So, first of all, are these options essentially the same thing? If not, what are the differences?
The persistent_term docs give a lot of warning about updating or deleting things, but would there be anything to be aware of if the term never ever changes?
If the term never changes use persistent term, it’s a great fit!
If it does change you can use the module recompilation hack that Discord use there. It’s a bit more compliacted how it works but unless you need to perform regular updates it’s not worth looking into it too much.
Problems between these approaches is mostly similar. In case of data that is set once and then never changes there will be no difference except the fact that persistent_term is easier to use.
The cost is the time it takes to update the otherwise almost-never-changing data piece. It can be punishing if you do it often.
Use :persistent_term for something that almost never will change. If you need application cache, use libraries that step on :ets (and add some more pleasantries) like Ane and Cachex.
If I remember the timeline correctly, the mochiglobal Erlang library was first (and fast_global is an Elixir port of this).
The persistent_term-module was added to Erlang as ‘blessed’ version of fast_global because multiple companies that were dealing with large amounts of data found out that this was a solution for a problem they were having.
This module exposes the same API, and internally works very similar, except that the functionality has been moved to ‘built-in functions’ and as such some optimizations can be done. For instance, it seems that persistent_term no longer re-compiles a whole module to store the new constants, but instead stores the contents in a (shared) hash-map.
Since it was released, multiple parts of Erlang itself that deal with configuration data that (almost) never changes also started to use the persistent_term functions for performance gains.
So if you’re on a new enough OTP version, it is almost surely a better idea to use persistent_term over the library-based implementations.
