Phillipp
Options for process state backup other than Postgres
Hey,
I wonder what would be some valuable options for process state backups if one does not want to add a database like Postgres to the setup.
In my particular case, I got a bunch of “tracking” processes that should survive an application restart (e.g. from deployment). Hot code reloading is not an option in our Kubernetes setup.
The state I need to backup does not contain Elixir specific data structures (e.g. atoms) so it can easily be encoded to JSON and back into lists and maps.
What I have thought of so far:
- Writing JSON files (e.g.
{identifier}.json) to disk and load them on startup - Writing state to Mnesia (maybe useful if we add clustering later)
Are there any other options I am not aware of? What would you guys do in my case?
First Post!
tty
I would favor mnesia and not bother with JSON since you can stuff the entire map/struct into mnesia without worrying about marshalling.
Most Liked
LostKobrakai
If you can store stuff on the filesystem I’d rather user :erlang.term_to_binary than json.
keathley
This point really can’t be emphasized enough. If you’re going to use mnesia you should be aware of its use cases and its limitations. Mnesia might be fine for your use case but you should really prove that first.
I like to use jsonb columns in postgres for these sorts of things but since you specifically called that out as a non-option then I might consider something like dynamodb. Dynamo is a reasonable choice if you just want to store binary blobs somewhere. Its hard to tell from your example but if you only need to read in the files once on app start then you might be able to store the files on s3 and pull them from there when the app is booting. There’s a lot of options here. Most importantly I would highly recommend avoiding creating an ad-hoc database if you can help it.
tristan
Postgres is likely the best way to go and is what I have used before, GitHub - erleans/erleans: Erlang Orleans · GitHub
Mnesia can be fine on a single node but, particularly if running in kubernetes, don’t expect to be able to utilize it as a distributed store.
A StatefulSet which ensures the same volume mounted for a specific nodename could be useful or storing a local copy. But far better to be able to have a stateless app deployment and use Postgres for data.
Popular in Discussions
Other popular topics
Chat & Discussions>Discussions
Latest on Elixir Forum
Sponsor Spotlight
We build reliable cloud platforms for business-critical systems.
Categories:
Sub Categories:
Forums
Popular Tags
- #ecto
- #liveview
- #troubleshooting
- #learning-elixir
- #deployment
- #library
- #erlang
- #testing
- #genserver
- #mix
- #absinthe
- #remote-other
- #otp
- #plug
- #how-to-question
- #macros
- #postgres
- #channels
- #elixirconf
- #exunit
- #discussion
- #code-sync
- #javascript
- #podcasts
- #onsite
- #dialyzer
- #docker
- #authentication
- #umbrella
- #full-time-contract
- #podcasts-by-brainlid
- #ecto-query
- #elixir-ls
- #phoenix_html
- #iex
- #blog-post
- #graphql
- #genstage
- #ai
- #websockets
- #supervisor
- #elixirconf-us
- #advent-of-code
- #distillery
- #processes
- #api
- #forms
- #metaprogramming
- #security
- #hex
Our Sponsors
Build Elixir applications with speed and confidence.
Supporting innovation across the BEAM ecosystem.
We build reliable cloud platforms for business-critical systems.
Catch errors, track performance, monitor hosts and more.
Error tracking, logs, uptime, and Just Enough APM. Built for developers.
Practical resources that improve the lives of professional developers.
Develop your skills with books, videos, and courses.
The deployment platform built for Elixir: PaaS ease, VPS control.
The team behind Membrane, Popcorn, LiveDebugger. Available for hire.
Courses that'll move you from confusion to "Aha, now I get it!"