sezaru
Oban jobs stuck as available if using global_limit
I have a queue that is configured like this:
queues: [
default: [
local_limit: 100,
global_limit: [
allowed: 4,
partition: [args: :tenant_id]
]
]
],
When I start a job in it, it never runs it, it will stuck in available state forever. But, if I remove the global_limit config and leave it with just the local_limit line, then the jobs run immediately.
I also noticed that, if I have the global_limit config enabled, Oban Web will not show any job at all, even if they do exist in the DB.
Not sure if it is related to the issue or not, but when I start my backend, I see this log message:
17:30:28.294 [info] {"message":"job staging switched to local mode. local mode polls for jobs for every queue; restore global mode with a functional notifier","mode":"local","source":"oban","event":"stager:switch"}
Here is my worker implementation:
defmodule Core.Workers.ApiCaller do
@moduledoc false
use Oban.Pro.Worker,
queue: :default,
recorded: [limit: 128_000_000]
require Logger
args_schema do
field :url, :string, required: true
field :args, :map, required: true
field :tenant_id, :integer, required: true
end
@impl Oban.Pro.Worker
def process(%Job{args: args} = _job) do
%{url: _url, args: _args, tenant_id: tenant_id} = args
dbg("start: #{inspect(self())} #{tenant_id}")
Process.sleep(5_000)
dbg("end: #{inspect(self())} #{tenant_id}")
{:ok, "output"}
end
end
Most Liked
sorentwo
Oh! That explains it. You’re using Core.Repo.insert!() rather than Oban.insert!(), so there’s no opportunity for Oban to calculate the partitioning key. You’d have the same problem with uniqueness, chains, etc. It should look like this:
def create! do
%{url: "http://localhost:4000/test_api", args: %{a: 1}, tenant_id: 1}
|> new()
|> Oban.insert!()
end
sorentwo
Which version of Pro are you using? This could be related to poor handling of missing partition keys in slightly older versions of Pro v1.6.
That’s not related, but it does indicate that you don’t have functional pubsub set up for that environment. It could be from a pooler like pgbouncer if you’re using the default Postgres notifier, or because you don’t have a functional cluster if you’re using the PG notifier.
Last Post!
sezaru
Ah, damn, that’s correct! I forgot that Oban.insert is not just a delegate to the Repo insert but actually to stuff behind the scene.. Now it is working great!
Thanks for the help!
Popular in Questions
Other popular topics
Latest Oban Threads
Categories:
Sub Categories:
Forums
Popular Tags
- #ecto
- #liveview
- #troubleshooting
- #learning-elixir
- #deployment
- #library
- #erlang
- #testing
- #genserver
- #mix
- #absinthe
- #remote-other
- #otp
- #plug
- #how-to-question
- #macros
- #postgres
- #channels
- #elixirconf
- #exunit
- #discussion
- #code-sync
- #javascript
- #podcasts
- #onsite
- #dialyzer
- #docker
- #authentication
- #umbrella
- #full-time-contract
- #podcasts-by-brainlid
- #ecto-query
- #elixir-ls
- #phoenix_html
- #iex
- #blog-post
- #graphql
- #genstage
- #ai
- #websockets
- #supervisor
- #elixirconf-us
- #advent-of-code
- #distillery
- #processes
- #forms
- #api
- #metaprogramming
- #hex
- #security









