sorentwo

sorentwo OP

Oban Core Team

The log size issue was due to advisory locks, which are no longer used, largely for that exact reason!

Advisory locks themselves are very lightweight, but they are owned by the connection that took the lock initially. When any other connection attempts to release the lock Postgres logs a big fat WARNING. This situation happens constantly because the connection checked out from the db pool is random—the larger the pool the less likely that the checked out connection owned the advisory lock.

The new system writes rows to an oban_beats table and operates warning free. Definitely upgrade to 0.8.0 if you haven’t already!

6
Post #92
AndyL

AndyL

Aah - didn’t read the notice!! Just upgraded. From midnight to 06:00 with old Oban, Postgres generated 13Gb of log data. After upgrade and watching the system for 30 minutes with two live apps, I see zero heartbeat data in the postgres logs. Yahoo! :slightly_smiling_face:

3
Post #93
sorentwo

sorentwo OP

Oban Core Team

Really pleased to hear that it worked as expected! :yellow_heart:

As a side benefit, the new approach is more reliable—if a producer crashes the jobs can be recovered right away on startup, whereas with advisory locks they wouldn’t be recovered until the system restarted. (I wrote about that in more detail on Slack through a thread with @chrismccord, but it isn’t available anymore).

0
Post #94
AndyL

AndyL

Same! Just FYI here’s a screenshot from one of my Oban apps - a WIP cron-style job scheduler used internally. Oban and LiveView work well together!

crow
crow793×883 105 KB

Here’s the source: https://github.com/andyl/crow

9
Post #95
typicalpixel

typicalpixel

Is there a way to use Oban.insert in a multi, with function passing in order to get a result from a previous insert’s result?

1
Post #96
LostKobrakai

LostKobrakai

Calling it within Ecto.Multi.run should work just fine.

1
Post #97
davit-khaburdzania

davit-khaburdzania

@sorentwo is there way to check out the Web UI, that is in development ?

1
Post #98
sorentwo

sorentwo OP

Oban Core Team

@LostKobrakai is correct, you can call it with Multi.run just fine. That’s all that Oban’s Multi support is really doing anyhow.

0
Post #99
typicalpixel

typicalpixel

Okay, thank you. That is what I was doing, and it’s working fine, but wasn’t sure if that was the correct method.

0
Post #100
sorentwo

sorentwo OP

Oban Core Team

As of yesterday there is: https://oban.dev/oban

It is definitely a work in progress with a lengthy list of things to fix and add. Now that the underlying db structures are stable and phoenix live view has been officially released I can make progress!

Note that this is something I’m hoping to control access to, so the GH repo isn’t public.

8
Post #101