Oban silently accepting failed jobs with suspicious return values

We recently had an authentification problem with an external service, and it went unnoticed, because we returned the clients return value unchanged from the job.

{:error, "some ID", %Tesla.Env{…}}

Oban silently accepted this jobs as “Done” which made them slip our monitoring and alerting.

Even though we now made sure that from this particular job there is no way to return an improper return value, there is now some uncertainty that this might happen again with other jobs, old ones as new ones.

Out of curiosity, even though we haven’t decided yet, whether false positives or false negatives do hurt more here, I am curious and therefore want to ask: is there a way to configure Oban to fail a job on unexpected return values?

It’s not silent, there’s a warning logged every time that happens:

If you have access to Pro, you could set up a global hook that verified the shape of returned values and failed more loudly (logged a message, sent a sentry report, etc).

Yeah, I have seen it in the logs when debugging the issue.

But now as you have pointed me again towards it, I will check whether we can make our alerting sensible to log messages of this format!