ModuleStore - use a compiled module as a high-performance key-value store

Is there a reason to not use the erlang builtin :persistent_term.

https://www.erlang.org/doc/apps/erts/persistent_term.html

I’m aware of :persistent_term it’s quite similar of course. The difference is that persistent term is global, and you have to namespace your items by using tuple keys. Here you can have one or more module stores with just your stuff, which is nice. Your code will read MySpecialStore.get(... and it’s absolutely the fastest possible thing to have, which is also nice. Apart from writes of course

To be clear modules are also global. If you want a similarly ergonomic API you could just write a wrapper function around the persistent term lookup. As someone’s done a lot with compiled modules in Absinthe persistent term is just so much better.

Thanks for the feedback and taking an interest can you elaborate on the problems you encountered? Do you have reason to believe this approach won’t work or cause problems under real world conditions? I know you can’t store everything this way, like refs.

I’m aware modules are global too. But we still consider MyAppWeb.Endpoint to be my endpoint, as opposed to Phoenix.Endpoint, and it has its own config. So I was thinking among those lines.

:persistent_term was added in response to people creating modules at runtime to store data. It explicitly is an api to replace what you’re doing with a native api, which gives the otp team a bit of control over the usecase over abusing unrelated apis around modules.

Namespacing shouldn’t really prevent you from using :persistent_term either.

Ah I hadn’t realized that. I thought I was doing something similar but with slightly different runtime characteristics. I figured it would be a nice little self-contained thing to contribute back, just trying to help out. It seems I’m reinventing the wheel in a way that is less round then the original so it’s probably best to pull the lib

Then again, I the benchmark results do return some different results for compiled module vs persistent_term, especially with regards to the standard deviation (if the difference is significant is another question of course). That’s why I figured there might be a case for it.

I guess my question is: will this go horribly wrong in some way that I’m missing? If so, I’m really not to proud to pull the lib. If not, then why can’t this be out there as an alternative, apart from the fact that this is not what the compiler is meant to do?

If we’re discussing abusing features of the language have you benchmarked against using the process dictionary? Using it for configs is specifically called out as a possible appropriate use case in this post:

Write-once process parameters . Think of this as an initial configuration step. Stuff all the settings that will never change within a process in the dictionary. From a programming point of view they’re just like constants, so no worries about side effects.

For some more prior art, also check out GitHub - discord/fastglobal: Fast no copy globals for Elixir & Erlang. · GitHub. But yeah, I believe persistent_term was designed to solve problems like these.

That’s really interesting, thanks for the link