Oban running on multiple nodes

Hi ,

My requirement is I have to publish messages to Kafka based on events raised by my phoenix app , I am evaluating Oban to be aligned with the outbox pattern , our application will be deployed in 3 nodes in production with a single postgres DB to read/write from , the “outbox” or the Oban’s Repo will be hosted in the same DB.

My doubt is when it comes to multi node where all the Oban workers from the different nodes will be reading from the same table . whats the behaviour as in when all the 3 workers will read from the same table , will there be any chances of clash as in when all the 3 workers are reading the same event to publish ?

Or such a scenario is not at all possible ?

If it is possible is there any workaround/pattern available to avoid the same ?

Workers will not process the same job simultaneously, so it is safe.

2 Likes

Thanks @egze , but how does it guarantee that , if you have a link to an article regarding it , that would help a lot.

You can see the code used to reserve jobs uses locks: https://github.com/sorentwo/oban/blob/master/lib/oban/query.ex#L21

This post explains the general technique:

3 Likes

Thanks a lot @mbuhot

1 Like

Is this a duplicate of Oban — Reliable and Observable Job Processing ?

Yeah Sorry for that @benwilson512