budgie
Best options for Elixir-only persistent distributed cache?
Building something that uses realtime features with liveview. There’s state that’s managed and pushed to call participants. Right now I’m using Cachex but if I do a deploy while someone is in a call, all the state will be lost?
What’s the best way to have temporary in-memory storage that persists across restarts without resorting to an external service i.e. memcached or redis?
Most Liked
Schultzer
dimitarvp
Got no answer for you at the moment but why are you using an ephemeral state that can be lost at any time (cache) as a source of truth / database?
lud
Depends on how you deploy.
You can build a distributed cache that synchronizes agressively, and when you shut down a node to roll a new node, the stopping node stops accepting external input but will wait till all its currently processing cache items are synced on other nodes, and the new node will not start to accept input before being in sync with all other nodes.
k8s can help with that, but it’s probably a lot of pain to do properly.
Otherwise you can use PubSub : all cache writes go to pubsub and are distributed to all other nodes. When a new node start, it subscribes to pubsub without processing messages yet, it selects another node and gets a full copy of the cache to bootstrap the cache locally, and then processes incoming messages. Writes must be time-stamped to ignore old writes (aso because the node you are copying from is already listening to messages sent during the boostrap sequence and the new node will receive them too. If you start listening after copying the cache you will miss some messages.).
All of this looks hard to get right, are you sure you cannot use Redis?
Last Post!
budgie
Yeah thanks for the reality check guys. I think redis is ultimately what I’ll go with. Love the idea of having them replicated cluster-internally but don’t want to have to add persistent block storage or a volume mount to the app containers. Keep Cachex for read-through queries to the DB probably.
Popular in Questions
Other popular topics
Latest on Elixir Forum
Sponsor Spotlight
Catch errors, track performance, monitor hosts and more.
Categories:
Sub Categories:
Forums
Popular Tags
- #ecto
- #liveview
- #troubleshooting
- #learning-elixir
- #deployment
- #library
- #erlang
- #testing
- #genserver
- #mix
- #absinthe
- #remote-other
- #otp
- #plug
- #how-to-question
- #macros
- #postgres
- #channels
- #elixirconf
- #exunit
- #discussion
- #code-sync
- #javascript
- #podcasts
- #onsite
- #dialyzer
- #docker
- #authentication
- #umbrella
- #full-time-contract
- #podcasts-by-brainlid
- #ecto-query
- #elixir-ls
- #phoenix_html
- #iex
- #blog-post
- #graphql
- #genstage
- #ai
- #websockets
- #supervisor
- #elixirconf-us
- #advent-of-code
- #distillery
- #processes
- #api
- #forms
- #metaprogramming
- #security
- #hex
Our Sponsors
Build Elixir applications with speed and confidence.
Supporting innovation across the BEAM ecosystem.
We build reliable cloud platforms for business-critical systems.
Catch errors, track performance, monitor hosts and more.
Error tracking, logs, uptime, and Just Enough APM. Built for developers.
Practical resources that improve the lives of professional developers.
Enabling companies to succeed by building software people love.
The deployment platform built for Elixir: PaaS ease, VPS control.
The team behind Membrane, Popcorn, LiveDebugger. Available for hire.
Courses that'll move you from confusion to "Aha, now I get it!"