tedi

tedi

Unique job handling

Hello,

I am using Oban [Pro] for all of my queues.

I have been hitting a wall on one of my queue functionalities. I have the following conditions:

  • Users update their data for a certain resource quite frequently. There are many instances where the user triggers an update action before a job completes. I have a unique clause by the keys user_id and resource_id
  • I need the job to execute one at a time when that uniqueness clause is true.
  • If a job with the user_id and resource_id is already running, I would like to queue the new job to run after the already running job is completed.
  • If this happens more than once and there’s already another job in the queue with the same keys, overwrite the one that’s queued to start next. Example:
    • Job 1 is executing
    • Job 2 with the same user_id and resource_id is queue to run next
    • We receive job 3 with the same user_id/resource_id. I need Job 2 to be replaced with job 3 (and job 4,5,6…etc do the same)

This is what I have:

      MyApp.Workers.MyJobWorker.new(
        args,
        queue: :test_queue,
        unique: [period: :infinity, fields: [:args], keys: [:user_id, :resource_id]],
        replace: [:args]
      )
      |> Oban.insert()

However with this set-up, the new jobs don’t get stored when there is a unique conflict.

Thanks,
Tedi

Marked As Solved

sorentwo

sorentwo

Oban Core Team

All you’re missing is setting the unique states. You only want to consider :available and :scheduled states for uniqueness (and maybe :retryable).

[
  period: :infinity,
  fields: [:args],
  keys: [:user_id, :resource_id],
  states: [:available, :scheduled]
]

Also Liked

tedi

tedi

Ah I see, that did it, thank you!

Just to confirm, for the config, I have the following set-up:

test_queue: [
          local_limit: 10,
          rate_limit: [allowed: 1, period: 30, partition: [fields: [:args], keys: [:user_id, :resource_id]]]
        ],

Seems to be fine from initial testing, but this would work globally across nodes as well, correct? Capping each job to run per user at 30 seconds.

sorentwo

sorentwo

Oban Core Team

That’s right. You’ve got it!

Where Next?

Popular in Questions Top

srinivasu
How to handle excepions in elixir? Suppose i have A, B, C ,D, E modules. and each module has get() function. A.get() method will call t...
New
myronmarston
The Elixir Typespec docs show the following syntax for keyword lists in typespecs: # ... | [key: type] # keyword lists...
New
vonH
In asking this question I am more interested about the expressiveness of the language itself and less concerned about the availability of...
New
hariharasudhan94
I would like to know what is the best IDE for elixir development?
New
earth10
Hi, I’m just starting to build a side-project with Elixir and Phoenix and doing some basic test with Elixir alone. What strikes me is th...
New
shahryarjb
Hello, I get Persian date from my client and convert it to normal calendar like this: def jalali_string_to_miladi_english_number(persi...
New
JDanielMartinez
Hi! May someone helps me, please! I have two apps into an umbrella project: the first one is Database, which manages queries, and the se...
New
pmjoe
I have a relationship of love and hate with Elixir. Lots of things are just absolutely right, but there are some things that are kind of ...
New
tduccuong
Hi, is there any work on GUI with Elixir, that is similar to Electron/Javascript? My idea is to bundle Phoenix and BEAM into a single se...
New
LegitStack
I’m trying to make a websocket server in Phoenix or raw Elixir. I heard about gun, I think I could use cowboy, but since I’m not that sma...
New

Other popular topics Top

Qqwy
Update: How to use the Blogs & Podcasts section You can post links to your blog posts or podcasts either in one of the Official Blog...
3271 126479 1222
New
JeremM34
Hello, how can I check the Phoenix version ? Thanks !
New
vegabook
I’m brand new to Phoenix and I have stripped one of the demo applications to the bone. I just want to get an svg up on the screen. Here i...
New
hariharasudhan94
lets say i have a sample like a = 20; b = 10; if (a > b) do {:ok, "a"} end if (a < b) do {:ok, b} end if (a == b) do {:ok, "equa...
New
Darmani72
If I have a post route which an argument: post /my_post_route/:my_param1, MyController.my_post_handler How would get the post params ...
New
aadeshere1
I have a another noob question about loop. Since elixir is immutable, while loop is not directly possible. total = 10 while total != 0 ...
New
Fl4m3Ph03n1x
About me? ( if you have nothing better to do than reading about some random guy in the internet :stuck_out_tongue: ) Hello all, this is ...
New
AstonJ
We’ve put together this wiki for Phoenix LiveView - please feel free to add any info you feel is worth including. What is Phoenix LiveV...
New
romenigld
I am trying to run a deploy with docker and I successfully runned with this command: docker build -t romenigld/blog-prod . but when I t...
New
Harrisonl
We have an ECS cluster with 4 services, where each task joins a single cluster, via discovery ECS discovery service. Currently when I de...
New

We're in Beta

About us Mission Statement