dhilipsiva
Hello Community,
I come from a predominantly python background and I am very new to elixir. So I am having a hard time figuring out some things in Elixir (Phoenix/Oban).
Recently, my company took over a Elixir product that someone else built. It was built with Phoenix and they have used Oban to run asynchronous tasks. These tasks are very compute intensive (they do a lot of calculations) and its slowing the APIs down.
If it was python, we usually have one command to run API server and another command to run background tasks (usually a celery command). So I build a separate “api” container and a separate “task” container. I just scale “task” container and it won’t slow down “api” containers as they run on independent machines.
I am having a hard time figuring out how to do the same thing with Pheonix/Oban. Basically, what I am looking for is a way to start API and Tasks separately (using two different commands/ passing some flags).
Plus I discovered they have used something called “libcluster” in the product (I have a basic idea. I know I might have to use a different clustering strategy when I do deployment). I am not sure if this is relevant, but I somehow felt that these things are connected.
Can someone please point me to relevant resources? I just want to be able to deploy API server & Oban (or quantum) tasks independently.
Where Next?
Trending in Questions
Other Trending Topics
Latest Phoenix Threads
Latest Oban Threads
Latest on Elixir Forum
Sponsor Spotlight
We build reliable cloud platforms for business-critical systems.
Categories:
Sub Categories:
Forums
Popular Tags
- #ecto
- #liveview
- #troubleshooting
- #learning-elixir
- #library
- #deployment
- #erlang
- #testing
- #genserver
- #mix
- #absinthe
- #remote-other
- #otp
- #plug
- #how-to-question
- #macros
- #postgres
- #elixirconf
- #channels
- #exunit
- #discussion
- #code-sync
- #podcasts
- #javascript
- #onsite
- #dialyzer
- #docker
- #authentication
- #umbrella
- #full-time-contract
- #podcasts-by-brainlid
- #ai
- #ecto-query
- #elixirconf-us
- #blog-post
- #elixir-ls
- #phoenix_html
- #iex
- #graphql
- #genstage
- #websockets
- #supervisor
- #advent-of-code
- #distillery
- #processes
- #elixirconf-eu
- #api
- #forms
- #metaprogramming
- #hex
Our Sponsors
Build Elixir applications with speed and confidence.
We build reliable cloud platforms for business-critical systems.
Catch errors, track performance, monitor hosts and more.
Error tracking, logs, uptime, and Just Enough APM. Built for developers.
Elixir, Erlang & BEAM conferences from Code Sync - join us in 2026!
Practical resources that improve the lives of professional developers.
Bringing learning, work and compliance together since 1987.
Producing high quality Elixir screencasts since 2017.
The team behind Membrane, Popcorn, LiveDebugger. Available for hire.
Courses that'll move you from confusion to "Aha, now I get it!"
Showing Posts 1 to 1- Show Best Posts
- Show All (oldest first)
- Show All (newest first)
sorentwo
You probably need to run Oban in all application instances. Most likely the API inserts Oban jobs, which requires a running instance (though it needn’t run any queues).
You can configure each application instance to run specific queues and avoid heavy computation on API instances. There is an official guide that covers this exact use case: Splitting Queues Between Nodes — Oban v2.23.0