Is it possible to have a different Repo for the Oban Postgres notifier?

Questions / Help
1

We’re using pgbouncer in front of Heroku Postgres (this runs in transactional mode) which means we can’t use the Oban notifier out of the box. I see that we could use the repeater plugin to solve this but I was hoping it might still be possible to get this working with the Postgres notifier.

To achieve this I believe I can create a new Repo module that connects directly to the database and configure Oban with this. However this would mean all of my Oban.inserts that aren’t part of a multi would also skip past pgbouncer which isn’t desirable for us.

Is it possible to have all Oban connections go through the pooling and have a separate connection for the notifier that is direct to the database?

2

I believe Oban has this Oban.Plugins.Repeater — Oban v2.8.0 for that situation.

3

Yes that’s correct and is what we’re currently using. I’m not completely sold on this approach however since it involves using a polling interval which gives me the impression we might be processing jobs slower compared to the postgres notifier approach.

4

That’s why the repeater plugin exists, for transaction pooling mode in pg_bouncer. There are three options:

  1. Switch pg_bouncer from transaction to session mode. There are tradeoffs, but everything in Oban will work without additional plugins.
  2. Switch to a different notifier module, like pg in Pro. This isn’t a viable option for you because you’re on Heroku and distributed Erlang doesn’t work. It also wouldn’t get trigger-based notifications, so you’d still be polling for new jobs.
  3. Use the repeater plugin as you are :+1:

The default polling interval is 1s. That will only make a difference for idle queues. If the queue has a continuous stream of jobs then the producer will keep fetching them when it has more availability, regardless of the polling interval.

2 Likes
5

The default polling interval is 1s . That will only make a difference for idle queues. If the queue has a continuous stream of jobs then the producer will keep fetching them when it has more availability, regardless of the polling interval.

I misunderstood how the polling was working, thanks for the clarification.

2 Likes