Example of multiple node worker cluster with Oban?

Does anyone know if there is an example repo or docs/guide on how to set up a cluster of multiple worker nodes using Oban? I have a use case similar to https://sorentwo.com/2019/11/05/oban-recipes-part-7-splitting-queues.html where I would like expensive GeoTIFF processing jobs to run on a separate node to my main web app’s node.

The article shows how you can use environment variables to configure each node’s queue concurrency at runtime which is great but how do you configure these nodes to talk to each other? The web app node will enqueue the job that should be performed by the GeoTIFF worker node. I don’t think Oban uses distributed erlang or PG2, etc. Do you simply attach the nodes to the same pgsql repo?

afaik - yes - it’s that simple…

2 Likes

Yes, this is all you need to do.

1 Like

Thanks @outlog and @benwilson512… that’s really great! I guess I should have experimented first before asking the question for some reason I just assumed wiring it up would be more complex than this.

2 Likes