MatheusBD15

MatheusBD15

Oban: how to list/count executing jobs in the current node

How do I list or count Oban jobs which are currently executing, but only in the same node my code is running?
I’ve tried running something like:

Repo.all(
      from(job in Job,
        where: job.worker == ^MyWorker,
        where: job.state in ["executing"],
      )
    )

But that gives me all the executing jobs across all nodes, and I need to know which jobs are executing only on the current node.

Currently, our backend runs in a kubernetes cluster, and this is related to some graceful shutdown logic, so we need it to be node specific. We also have Oban Pro.

Marked As Solved

sorentwo

sorentwo

Oban Core Team

The first element is the node name, but the second is the queue producer’s UUID. You don’t need the UUID to query for jobs from the current node.

Some of Oban Web’s filter queries show how you can query by the current node: oban_web/lib/oban/web/queries/job_query.ex at main · oban-bg/oban_web · GitHub. It’s essentially a dynamic containment operator:

dynamic([j], ^condition and fragment("?[1]", j.attempted_by) in ^nodes)

Here’s how you’d translate that into a general query for the current node:

# Use the running config to get the node name.
conf = Oban.config()

Oban.Job
|> where([j], j.state == "executing" and j.worker == MyWorker)
|> where([j], fragment("?[1]", j.attempted_at) == ^conf.node)
|> Repo.all()

Rather than building your own query, you could use Oban.Met.latest/3 to get the executing count:

%{"all" => count} = Oban.Met.latest(Oban, :exec_count, filters: [node: "my-node-name"])

Also Liked

MatheusBD15

MatheusBD15

Hey Soren!
Thanks a lot for the answer, I appreciate it.
The first option, with writing a query, worked very well!

# Use the running config to get the node name.
conf = Oban.config()

Oban.Job
|> where([j], j.state == "executing")
|> where([j], fragment("?[1]", j.attempted_by) == ^conf.node)
|> Repo.all()

For curiosity’s sake, using

Oban.Met.latest(Oban, :exec_count, filters: [node: "my-node-name"])

always gives me an empty map as a result.

However,

Oban.Met.latest(Oban, :full_count, filters: [node: "my-node-name", state: "executing"])

always gives the correct result! Why would that be?
Does it happen that a Job has the “executing” state, but it does not appear in the exec_count gauge?

Edit:
I think I understand it. It’s due to a misunderstanding of what the job state means. A Job might currently have the “executing” state, but not be running, in fact, it can mean that the job will execute in the future, is this correct? Or it might just be my database being inconsistent.

sorentwo

sorentwo

Oban Core Team

That’s most likely what is happening. Jobs reported by exec_count have live processes running. It’s possible to have job records marked as executing in the database when the processes aren’t running anymore. Those are called orphaned jobs, and they typically happen during shutdown when jobs run longer than the queue shutdown period.

There’s more about that situation in the Ready for Production guide.

It doesn’t mean that the job may run in the future. Most likely, it was running in the past and the queue shut down.

Where Next?

Popular in Questions Top

sen
Hi All, I set a environment variables in dev.exs , like below code. when i start server, how can i set the ${enable} value? thanks. d...
New
Darmani72
If I have a post route which an argument: post /my_post_route/:my_param1, MyController.my_post_handler How would get the post params ...
New
gshaw
What is the idiomatic way of matching for not nil in Elixir? E.g., First way: defp halt_if_not_signed_in(conn, signed_in_account) when...
New
fireproofsocks
I’m working on defining a simple Ecto schema for a table (in PostGres), but I don’t see where I can define a column as NOT NULL. Conside...
New
baxterw3b
Hi guys, i’m new in the Elixir world, and i have to say, that i love it! i’m having some problem to understand anonymous functions with ...
New
vonH
When I run the Plug and I recompile I wind up having to use Ctrl C to quit iex and start again. Witht the help of rlwrap I can use the cu...
New
sergio_101
I am VERY much an elixir newbie. I have taken one elixir course and one phoenix course on Udemy. During that course, I saw the instructor...
New
Qqwy
Original source of discussion: This topic on the Pragmatic Programmers’ Functional Web Development with Elixir, OTP, and Phoenix forum. ...
New
dotdotdotPaul
Okay, I’m having a heck of a time trying to figure out how to best handle the validation of belongs_to associations in Ecto. I’m sure I’...
New
joaquinalcerro
Hi there, I am working with Ecto-Postgresql and I need to call all of the records from a specific table but the table has 40,000 records...
New

Other popular topics Top

johnnyicon
Hi all, I’ve just started learning Elixir and Phoenix Framework, so please pardon my n00bness at this stage. I’m trying to use Postgres...
New
Lily
In templates/appointment/index.html.eex: <%= for appointment <- @appointments do %> <tr> <td><%= appoi...
New
aesmail
Hello guys, I have finally made it. I created an admin interface for a framework. It’s been on my todo list for years and with the curre...
New
vrod
I am using the Starship cross-shell prompt – it seems pretty nice, but I get some errors: [WARN] - (starship::utils): Executing command ...
New
alice
Hey, Just curious what are the main benefits of Elixir compared to Clojure? When is Elixir more useful than Clojure and vice versa? Th...
New
freewebwithme
Using vs code and installed ElixirLS: support and debugger. And I got an error popped up on start up says Failed to run ‘elixir’ comma...
New
shijith.k
I am trying to start a new phoenix project with elixir 1.9, but mix phx.new does not work. It says that ** (Mix) The task "phx.new" could...
New
WestKeys
Currently suffering from paralysis by [HTTP client] analysis. This is rather unusual in Elixirland as there tends to be consensus on the ...
New
openscript
Hello! Sorry for this astonishing simple question, but I’m really stuck. I try to set up the intellij-elixir plugin, but I don’t know ho...
New
sergio
Kind of like when jquery came out, it was super necessary. Existing drag and drop libraries have a bunch of baggage to support old browse...
New

We're in Beta

About us Mission Statement