tmock12
Replacing Rails Background Jobs with an Elixir Service
Need opinions on something. One of our projects is a large Rails monolith with a ridiculous amount of background jobs. We keep creating additional jobs and adding features to them. More and more I find myself thinking “… if this was Elixir we could do this and that more efficiently…“. I’ve seen talks in the past about replacing parts of their Rails ecosystem with Elixir services.
Has anyone done this as a goal of replacing background jobs with Elixir processes? Any thoughts on that experience?
Doing a full Rails replacement would be wonderful, but is unfortunately not feasible at this point. Is introducing a new tech stack worth it? Would love any opinions from anyone that has introduced Elixir as a service to their existing apps.
Thanks!
Most Liked
hubertlepicki
Yes, that’s precisely why doing events / reactions versus background jobs is better approach IMHO. You don’t schedule a job, you send out an event somewhere. That somewhere is a queue. Then some things can pick up that event and act upon it. The major advantage of thinking this way is that you are detaching the event from implementation. If you save to database module name / function / arguments to be executed, you are also prone to the code changes - i.e. suddenly the module is not available after deploy, or it’s taking now 3 not parameters and your background queue is going to get messed up.
If you are writing your Rails app properly, i.e. there is not much logic in the models themselves, and you have stuff extracted to services, you can quite easily connect from Ecto to the same database Rails uses. It’ll understand it well, and can manipulate the same data Rails does. You obviously might need to re-implement some of the logic in Elixir but generally that’s what I would also do in Rails - background job services would be implementing own logic wherever possible to keep it isolated.
axelson
In case anyone is still coming to this thread (based on views it appears so). Another good option for running persistent background jobs in Elixir is GitHub - oban-bg/oban: 💎 Robust job processing in Elixir, backed by modern PostgreSQL, SQLite3, and MySQL · GitHub (this is the job library that I like to use).
anthonator
If you’re wondering if you can do this purely in Elixir, you can. However, you are going to need to roll your sleeves up and get dirty learning OTP. Personally, I think implementing a job queue/event dispatcher is a great way to learn Elixir/OTP. The use case hits on most of the OTP sweet spots and requires you to dig a little.
You’ll need to put some effort in up front to understand how advanced you want your queue to be. Specifically, how durable should your jobs be and should they be distributed? If your server crashes or restarts is it OK to lose all your jobs? Do you need/want multiple servers handling jobs? If you need durability and distribution then you’re probably better off reaching for something like Exq or Honeydew. If not then Task’s might be a good starting point.
Popular in Questions
Other popular topics
Categories:
Sub Categories:
Forums
Popular Tags
- #ecto
- #liveview
- #troubleshooting
- #learning-elixir
- #deployment
- #library
- #erlang
- #testing
- #genserver
- #mix
- #absinthe
- #remote-other
- #otp
- #plug
- #how-to-question
- #macros
- #postgres
- #channels
- #elixirconf
- #exunit
- #discussion
- #code-sync
- #javascript
- #podcasts
- #onsite
- #dialyzer
- #docker
- #authentication
- #umbrella
- #full-time-contract
- #podcasts-by-brainlid
- #ecto-query
- #elixir-ls
- #phoenix_html
- #iex
- #blog-post
- #graphql
- #genstage
- #ai
- #websockets
- #supervisor
- #advent-of-code
- #elixirconf-us
- #distillery
- #processes
- #forms
- #api
- #metaprogramming
- #security
- #performance








