Cross-posting from slack here: In my first project that uses Oban, I found myself writing a GenericWorker module that takes as arguments an MFA tuple and calls Kernel.apply on that.
Now I see upsides as well as downsides to this - it feels like I’m freed from having to define a worker for things that don’t require much oban awareness, but on the other hand perhaps I’m kidding myself that it’s a good idea - I’ve already found 2 cases where I’m backing out the GenericWorker in favour of specific worker implementations.
Opinions and advice on the usefulness of generic Oban workers welcome, along with any tips on organising worker code.
You don’t have to get fancy with this. Oban is very approachable, you basically make one module, do use Oban.Worker, queue: :send_stuff inside, have a def perform(%Oban.Job{args: args}) do #... function definition inside and you’re pretty much set with your worker.
Then configure various parameters inside your config/*.exs files. A complete example that’s more than enough:
That’s it really. There is no need to abstract stuff. Not that that should stop you from having a generic worker, of course, but in my experience requirements very quickly make it mandatory to have specialized workers. But for your use-case it might work just fine.
At my previous job we used to have something like that for two particular use cases.
The first use case was to contain and granularly execute data migration. We were migrating data from MongoDB to Postgres and normalizing data through this process, beacuse of that we coudn’t trust transactions. Because of that there was a lot of changes on what becomes a job and what is done in bulk operations. this was the reason for having a generic worker to handle overall data migration.
The other use case was to deal with a registration process on a third party that we didn’t control the updates and that their webhooks weren’t reliable. because of that we modeled a state machine as a recurring job so it could check through the api the current state of the registration and update the state.
In both cases the approach helped reducing the boilerplate required to implement those features what helped us to deliver it faster. The downside is that you need to have a good instrumentation and be thoughtful of how you log what is going on inside the job and what is the context of errors that happens, just surfacing errors and trusting the stored error in oban isn’t enough. A dedicated worker limits the scope of the job that makes debugging, logging and understanding failures way more easy.
