dhilipsiva
Hello Community,
I come from a predominantly python background and I am very new to elixir. So I am having a hard time figuring out some things in Elixir (Phoenix/Oban).
Recently, my company took over a Elixir product that someone else built. It was built with Phoenix and they have used Oban to run asynchronous tasks. These tasks are very compute intensive (they do a lot of calculations) and its slowing the APIs down.
If it was python, we usually have one command to run API server and another command to run background tasks (usually a celery command). So I build a separate “api” container and a separate “task” container. I just scale “task” container and it won’t slow down “api” containers as they run on independent machines.
I am having a hard time figuring out how to do the same thing with Pheonix/Oban. Basically, what I am looking for is a way to start API and Tasks separately (using two different commands/ passing some flags).
Plus I discovered they have used something called “libcluster” in the product (I have a basic idea. I know I might have to use a different clustering strategy when I do deployment). I am not sure if this is relevant, but I somehow felt that these things are connected.
Can someone please point me to relevant resources? I just want to be able to deploy API server & Oban (or quantum) tasks independently.
Where Next?
Trending in Questions
Other Trending Topics
Latest Phoenix Threads
Latest Oban Threads
Latest on Elixir Forum
Sponsor Spotlight
Error tracking, logs, APM, uptime and cron monitoring in one dev-friendly platform. Find and fix problems before your users notice.
Categories:
Sub Categories:
Forums
Popular Tags
- #ecto
- #liveview
- #troubleshooting
- #learning-elixir
- #deployment
- #library
- #erlang
- #testing
- #genserver
- #mix
- #absinthe
- #remote-other
- #otp
- #plug
- #how-to-question
- #macros
- #postgres
- #channels
- #elixirconf
- #exunit
- #discussion
- #code-sync
- #javascript
- #podcasts
- #onsite
- #dialyzer
- #docker
- #authentication
- #umbrella
- #full-time-contract
- #podcasts-by-brainlid
- #ecto-query
- #elixir-ls
- #blog-post
- #phoenix_html
- #iex
- #graphql
- #ai
- #genstage
- #elixirconf-us
- #websockets
- #supervisor
- #advent-of-code
- #distillery
- #processes
- #api
- #forms
- #metaprogramming
- #security
- #hex
Our Sponsors
Build Elixir applications with speed and confidence.
Supporting innovation across the BEAM ecosystem.
We build reliable cloud platforms for business-critical systems.
Catch errors, track performance, monitor hosts and more.
Error tracking, logs, uptime, and Just Enough APM. Built for developers.
Producing high quality Elixir screencasts since 2017.
Enabling companies to succeed by building software people love.
The deployment platform built for Elixir: PaaS ease, VPS control.
The team behind Membrane, Popcorn, LiveDebugger. Available for hire.
Courses that'll move you from confusion to "Aha, now I get it!"
First Post!
sorentwo
You probably need to run Oban in all application instances. Most likely the API inserts Oban jobs, which requires a running instance (though it needn’t run any queues).
You can configure each application instance to run specific queues and avoid heavy computation on API instances. There is an official guide that covers this exact use case: Splitting Queues Between Nodes — Oban v2.23.0