tedi
Unique job handling
Hello,
I am using Oban [Pro] for all of my queues.
I have been hitting a wall on one of my queue functionalities. I have the following conditions:
- Users update their data for a certain resource quite frequently. There are many instances where the user triggers an update action before a job completes. I have a unique clause by the keys user_id and resource_id
- I need the job to execute one at a time when that uniqueness clause is true.
- If a job with the user_id and resource_id is already running, I would like to queue the new job to run after the already running job is completed.
- If this happens more than once and there’s already another job in the queue with the same keys, overwrite the one that’s queued to start next. Example:
- Job 1 is executing
- Job 2 with the same user_id and resource_id is queue to run next
- We receive job 3 with the same user_id/resource_id. I need Job 2 to be replaced with job 3 (and job 4,5,6…etc do the same)
This is what I have:
MyApp.Workers.MyJobWorker.new(
args,
queue: :test_queue,
unique: [period: :infinity, fields: [:args], keys: [:user_id, :resource_id]],
replace: [:args]
)
|> Oban.insert()
However with this set-up, the new jobs don’t get stored when there is a unique conflict.
Thanks,
Tedi
Marked As Solved
sorentwo
Oban Core Team
All you’re missing is setting the unique states. You only want to consider :available and :scheduled states for uniqueness (and maybe :retryable).
[
period: :infinity,
fields: [:args],
keys: [:user_id, :resource_id],
states: [:available, :scheduled]
]
1
Also Liked
tedi
Ah I see, that did it, thank you!
Just to confirm, for the config, I have the following set-up:
test_queue: [
local_limit: 10,
rate_limit: [allowed: 1, period: 30, partition: [fields: [:args], keys: [:user_id, :resource_id]]]
],
Seems to be fine from initial testing, but this would work globally across nodes as well, correct? Capping each job to run per user at 30 seconds.
1
sorentwo
Oban Core Team
Last Post!
tedi
Popular in Questions
I have VueJS GUIs with the project generated using Webpack.
I have Elixir modules that will need to be used by the VueJS GUIs.
I forese...
New
To my knowledge, put_in, Map.update etc. all have the one limitation of not automatically creating intermediate keys when needed (for exa...
New
Forgive me if this is obvious, but how does one delete a database record WITHOUT selecting it first? Ecto.Repo — Ecto v3.14.0 has exampl...
New
Hi everyone,
I was playing with phoenix liveView but I run into an issue. I have a form and want to validate each input text when the te...
New
I tried installing
elixir 1.11.2
erlang 23.3.4
via asdf in my zsh shell. Enabled the versions locally and globally.
When I list them ...
New
I have a server on AWS, and was running a load test using artillery. When looking at the Phoenix dashboard I see the Ports going to 100% ...
New
In the code below, if the create action is not set to accept “extra_key” as an input, it errors out with a message shown above. Is there ...
New
Other popular topics
Lets say I have map like this fetching from my database
%{"_id" => #BSON.ObjectId<58eb1a7a9ad169198c3dXXXX>, "email" => ...
New
For some reason my phoenix channels are working for me in my local dev environment, but as soon as I deploy via Docker, I get a 403 error...
New
Hi everyone!
I need implement if…else if…else condition from my elixir code, and anymore of this control flow structures not work proper...
New
I am VERY much an elixir newbie. I have taken one elixir course and one phoenix course on Udemy. During that course, I saw the instructor...
New
Surface is an experimental library built on top of Phoenix LiveView and its new LiveComponent API that aims to provide a more declarative...
New
I have a server on AWS, and was running a load test using artillery. When looking at the Phoenix dashboard I see the Ports going to 100% ...
New
Categories:
Sub Categories:
Forums
Popular Tags
- #ecto
- #liveview
- #troubleshooting
- #learning-elixir
- #deployment
- #library
- #erlang
- #testing
- #genserver
- #mix
- #absinthe
- #remote-other
- #otp
- #plug
- #how-to-question
- #macros
- #postgres
- #channels
- #elixirconf
- #exunit
- #discussion
- #code-sync
- #javascript
- #podcasts
- #onsite
- #dialyzer
- #docker
- #authentication
- #umbrella
- #full-time-contract
- #podcasts-by-brainlid
- #ecto-query
- #elixir-ls
- #phoenix_html
- #iex
- #blog-post
- #graphql
- #genstage
- #ai
- #websockets
- #supervisor
- #elixirconf-us
- #advent-of-code
- #distillery
- #processes
- #api
- #forms
- #metaprogramming
- #security
- #hex









