spacebat

spacebat

Advice on the use of generic Oban workers

Cross-posting from slack here: In my first project that uses Oban, I found myself writing a GenericWorker module that takes as arguments an MFA tuple and calls Kernel.apply on that.

Now I see upsides as well as downsides to this - it feels like I’m freed from having to define a worker for things that don’t require much oban awareness, but on the other hand perhaps I’m kidding myself that it’s a good idea - I’ve already found 2 cases where I’m backing out the GenericWorker in favour of specific worker implementations.

Opinions and advice on the usefulness of generic Oban workers welcome, along with any tips on organising worker code.

Most Liked

dimitarvp

dimitarvp

You don’t have to get fancy with this. Oban is very approachable, you basically make one module, do use Oban.Worker, queue: :send_stuff inside, have a def perform(%Oban.Job{args: args}) do #... function definition inside and you’re pretty much set with your worker.

Then configure various parameters inside your config/*.exs files. A complete example that’s more than enough:

config :my_app, Oban, repo: Oban.Repo, queues: [send_stuff: 50]

That’s it really. There is no need to abstract stuff. Not that that should stop you from having a generic worker, of course, but in my experience requirements very quickly make it mandatory to have specialized workers. But for your use-case it might work just fine.

cevado

cevado

At my previous job we used to have something like that for two particular use cases.

The first use case was to contain and granularly execute data migration. We were migrating data from MongoDB to Postgres and normalizing data through this process, beacuse of that we coudn’t trust transactions. Because of that there was a lot of changes on what becomes a job and what is done in bulk operations. this was the reason for having a generic worker to handle overall data migration.

The other use case was to deal with a registration process on a third party that we didn’t control the updates and that their webhooks weren’t reliable. because of that we modeled a state machine as a recurring job so it could check through the api the current state of the registration and update the state.

In both cases the approach helped reducing the boilerplate required to implement those features what helped us to deliver it faster. The downside is that you need to have a good instrumentation and be thoughtful of how you log what is going on inside the job and what is the context of errors that happens, just surfacing errors and trusting the stored error in oban isn’t enough. A dedicated worker limits the scope of the job that makes debugging, logging and understanding failures way more easy.

Where Next?

Popular in Questions Top

_russellb
I want to try my hand at web scraping. What tools/libraries do I need to use. I’m hoping to turn this into something professional so don’...
New
senggen
Erlang/OTP 25 [erts-13.2.2] [source] [64-bit] [smp:8:8] [ds:8:8:10] [async-threads:1] 15:22:35.803 [error] gen_event {lager_file_backend...
New
Harrisonl
We have an ECS cluster with 4 services, where each task joins a single cluster, via discovery ECS discovery service. Currently when I de...
New
qwerescape
Is there a way to get the call stack or stack trace at any point in the code? Not from exceptions, but an expression that returns how the...
New
fireproofsocks
I’m working on defining a simple Ecto schema for a table (in PostGres), but I don’t see where I can define a column as NOT NULL. Conside...
New
ashish173
I am using Ecto timestamps with postgres, I can see the timestamps() use the :naive_dateime but for my use case I wanted to store the ti...
New
nsuchy
Hi. I’ve noticed that Windows Powershell has it’s own IEX command and you cannot access Elixir’s IEX due to the conflict. This isn’t a cr...
New
joaquinalcerro
Hi there, I am working with Ecto-Postgresql and I need to call all of the records from a specific table but the table has 40,000 records...
New
dotdotdotPaul
Okay, I’m having a heck of a time trying to figure out how to best handle the validation of belongs_to associations in Ecto. I’m sure I’...
New
hariharasudhan94
Lets say I have map like this fetching from my database %{"_id" => #BSON.ObjectId<58eb1a7a9ad169198c3dXXXX>, "email" => ...
New

Other popular topics Top

senggen
Erlang/OTP 25 [erts-13.2.2] [source] [64-bit] [smp:8:8] [ds:8:8:10] [async-threads:1] 15:22:35.803 [error] gen_event {lager_file_backend...
New
Harrisonl
We have an ECS cluster with 4 services, where each task joins a single cluster, via discovery ECS discovery service. Currently when I de...
New
gshaw
What is the idiomatic way of matching for not nil in Elixir? E.g., First way: defp halt_if_not_signed_in(conn, signed_in_account) when...
New
chrismccord
Phoenix 1.4.0 released Phoenix 1.4 is out! This release ships with exciting new features, most notably with HTTP2 support, improved deve...
688 31194 112
New
minhajuddin
I have seen a lot of code which picks the first element from a list using Enum.at(0) instead of List.first. Is there a reason why people ...
New
Lily
In templates/appointment/index.html.eex: <%= for appointment <- @appointments do %> <tr> <td><%= appoi...
New
AstonJ
Please see the new poll here: Which code editor or IDE do you use? (Poll) (2022 Edition) It’s been a while since we first asked this, I...
208 31307 143
New
romenigld
I am trying to run a deploy with docker and I successfully runned with this command: docker build -t romenigld/blog-prod . but when I t...
New
klo
Got a question about when to concat vs. prepending items to list then reversing to achieve appending. So i know lists boil down to [1 | ...
New
komlanvi
Hi everyone, I was playing with phoenix liveView but I run into an issue. I have a form and want to validate each input text when the te...
New

We're in Beta

About us Mission Statement