Separating API from background tasks (oban/quantum) for deployment

Hello Community,

I come from a predominantly python background and I am very new to elixir. So I am having a hard time figuring out some things in Elixir (Phoenix/Oban).

Recently, my company took over a Elixir product that someone else built. It was built with Phoenix and they have used Oban to run asynchronous tasks. These tasks are very compute intensive (they do a lot of calculations) and its slowing the APIs down.

If it was python, we usually have one command to run API server and another command to run background tasks (usually a celery command). So I build a separate “api” container and a separate “task” container. I just scale “task” container and it won’t slow down “api” containers as they run on independent machines.

I am having a hard time figuring out how to do the same thing with Pheonix/Oban. Basically, what I am looking for is a way to start API and Tasks separately (using two different commands/ passing some flags).

Plus I discovered they have used something called “libcluster” in the product (I have a basic idea. I know I might have to use a different clustering strategy when I do deployment). I am not sure if this is relevant, but I somehow felt that these things are connected.

Can someone please point me to relevant resources? I just want to be able to deploy API server & Oban (or quantum) tasks independently.