daniel_torres
Hi there,
I’m currently investigating Elixir and Phoenix as an alternative to Ruby/Rails for a new project I’m working on.
It’s mainly an API backend and a mobile frontend, with websocket connections between the two, but most data is going through the JSON API.
However, learning more and more Elixir and OTP, I realised we could use Tasks and GenServers for “background jobs” instead of a full fledged job queue as you’d normally do with Rails. Background jobs in our case being things like:
- push notifications send to the mobile clients
- emails
- after-save calculations for our algorithms
- image processing
- search indexing
Now to the real question(s): say we’d use GenServers (potentially together with ETS/Mnesia) to do those things. If we deploy using erlang/elixir releases, the state in the GenServers should persists as far as I understand it, so we shouldn’t lose any unprocessed jobs. Correct?
However, if we deploy using throwaway docker containers, those unprocessed jobs would be gone after deploys.
Long story short, how would you guys handle such a thing? The aforementioned jobs are not super critical (a missed push notification is not the end of the world), but we’d certainly prefer a stable solution. Docker would also be nice from a DevOps perspective, that’s why I’m asking.
We could of course use an external queue with redis/rabbit, but that doesn’t feel like the erlang way, does it?
Hope that’s somewhat clear - thanks in advance!
Where Next?
Trending in Questions
Other Trending Topics
Latest on Elixir Forum
Sponsor Spotlight
Build Elixir applications with speed and confidence.
Categories:
Sub Categories:
Forums
Popular Tags
- #ecto
- #liveview
- #troubleshooting
- #learning-elixir
- #library
- #deployment
- #erlang
- #testing
- #genserver
- #mix
- #absinthe
- #remote-other
- #otp
- #plug
- #how-to-question
- #macros
- #postgres
- #elixirconf
- #channels
- #exunit
- #discussion
- #code-sync
- #podcasts
- #javascript
- #onsite
- #dialyzer
- #docker
- #authentication
- #umbrella
- #full-time-contract
- #podcasts-by-brainlid
- #ai
- #ecto-query
- #elixirconf-us
- #blog-post
- #elixir-ls
- #phoenix_html
- #iex
- #graphql
- #genstage
- #websockets
- #supervisor
- #advent-of-code
- #distillery
- #processes
- #elixirconf-eu
- #api
- #forms
- #metaprogramming
- #hex
Our Sponsors
Build Elixir applications with speed and confidence.
Supporting innovation across the BEAM ecosystem.
We build reliable cloud platforms for business-critical systems.
Catch errors, track performance, monitor hosts and more.
Elixir, Erlang & BEAM conferences from Code Sync - join us in 2026!
Practical resources that improve the lives of professional developers.
Master modern tools, write cleaner code, and tackle real-world challenges.
Bringing learning, work and compliance together since 1987.
The team behind Membrane, Popcorn, LiveDebugger. Available for hire.
Courses that'll move you from confusion to "Aha, now I get it!"
First Post!- Show Best Posts
- Show All (oldest first)
- Show All (newest first)
shavit
One solution is not to use background jobs in Elixir / Phoenix.
Let’s say you have a program that runs from a machine on a private network, then you can write a worker that subscribe to a job queue using Redis, and create jobs from the front end with Elixir / Phoenix.
If you don’t want to install another services like RabbitMQ or Redis, you can still use PostgreSQL or MongoDB with capped collections.
Most Liked
Qqwy
By the way, it probably is very possible to create a queue inside Elixir using the new GenStage/Flow. One possibility here would be to have a ProducerConsumer that will store its contents to the disk when its subscribers are not yet ready for the new requests.