sezaru

sezaru

Oban jobs stuck as available if using global_limit

I have a queue that is configured like this:

  queues: [
    default: [
      local_limit: 100,
      global_limit: [
        allowed: 4,
        partition: [args: :tenant_id]
      ]
    ]
  ],

When I start a job in it, it never runs it, it will stuck in available state forever. But, if I remove the global_limit config and leave it with just the local_limit line, then the jobs run immediately.

I also noticed that, if I have the global_limit config enabled, Oban Web will not show any job at all, even if they do exist in the DB.

Not sure if it is related to the issue or not, but when I start my backend, I see this log message:

17:30:28.294 [info] {"message":"job staging switched to local mode. local mode polls for jobs for every queue; restore global mode with a functional notifier","mode":"local","source":"oban","event":"stager:switch"}

Here is my worker implementation:

defmodule Core.Workers.ApiCaller do
  @moduledoc false

  use Oban.Pro.Worker,
    queue: :default,
    recorded: [limit: 128_000_000]

  require Logger

  args_schema do
    field :url, :string, required: true
    field :args, :map, required: true
    field :tenant_id, :integer, required: true
  end

  @impl Oban.Pro.Worker
  def process(%Job{args: args} = _job) do
    %{url: _url, args: _args, tenant_id: tenant_id} = args

    dbg("start: #{inspect(self())} #{tenant_id}")

    Process.sleep(5_000)

    dbg("end: #{inspect(self())} #{tenant_id}")

    {:ok, "output"}
  end
end

Most Liked

sorentwo

sorentwo

Oban Core Team

Oh! That explains it. You’re using Core.Repo.insert!() rather than Oban.insert!(), so there’s no opportunity for Oban to calculate the partitioning key. You’d have the same problem with uniqueness, chains, etc. It should look like this:

  def create! do
    %{url: "http://localhost:4000/test_api", args: %{a: 1}, tenant_id: 1}
    |> new()
    |> Oban.insert!()
  end
sorentwo

sorentwo

Oban Core Team

Which version of Pro are you using? This could be related to poor handling of missing partition keys in slightly older versions of Pro v1.6.

That’s not related, but it does indicate that you don’t have functional pubsub set up for that environment. It could be from a pooler like pgbouncer if you’re using the default Postgres notifier, or because you don’t have a functional cluster if you’re using the PG notifier.

Last Post!

sezaru

sezaru

Ah, damn, that’s correct! I forgot that Oban.insert is not just a delegate to the Repo insert but actually to stuff behind the scene.. Now it is working great!

Thanks for the help!

Where Next?

Popular in Questions Top

vonH
When I run the Plug and I recompile I wind up having to use Ctrl C to quit iex and start again. Witht the help of rlwrap I can use the cu...
New
Emily
I have VueJS GUIs with the project generated using Webpack. I have Elixir modules that will need to be used by the VueJS GUIs. I forese...
New
Brian
What is the proper way to load a module from a file in to IEX? In the python world, doing something like this pretty standard: from ....
New
joeerl
Hello again - after a longish gap I’ve decided I really must dig into Elixir and see what’s been happening here - so I have a few questio...
New
sen
Hi All, I set a environment variables in dev.exs , like below code. when i start server, how can i set the ${enable} value? thanks. d...
New
belgoros
I’m not a pro in using Regex and can’t figure out why the following behaviour happens, especially if we take into account the difference ...
New
romenigld
I am trying to run a deploy with docker and I successfully runned with this command: docker build -t romenigld/blog-prod . but when I t...
New

Other popular topics Top

rms.mrcs
Hi, I need to transform a list of numbers into a map where the keys are the indexes and the values are the original values of the list. ...
New
JeremM34
Hello, how can I check the Phoenix version ? Thanks !
New
vertexbuffer
Hello, can anybody help here..? I have a list of players and I what to delete an element, but every for loop the list is reverting to ori...
New
AstonJ
Seen any cool LiveView demos, sample apps or examples? Please post them here! :003:
New
shijith.k
I am trying to start a new phoenix project with elixir 1.9, but mix phx.new does not work. It says that ** (Mix) The task "phx.new" could...
New
dblack
I’ve got an issue with an app and I’ve no idea of how to troubleshoot it. I’m hoping someone here might have seen something similar. I p...
New

We're in Beta

About us Mission Statement