Are batch workers ran on the same node?

Hi!

At my company we have a deployment of Oban.Pro and we are heavily using Batch Workers.
We are running with 2 nodes but it’s scalable via HPA on our k8s deployment.

I wanted to know if there is any mechanism that assures that Batch workers will run on the same elixir node. Further, if they are ran on the same node will the callback worker also be assured to run on that same node?

I tried to find this in the Oban documentation but was unable to find an answer.

Thanks!

No, there isn’t a mechanism for automatic node affinity yet. Jobs are distributed between all nodes that run their queue. If you’d like to restrict a set of jobs to a single node, it’s best to only run queues on that node.

You may be interested in the only option available in the DynamicQueues plugin.

3 Likes