jonbo

jonbo

Oban/Telemetry error handling not capturing exceptions

Hi all,

Trying to figure out exceptions raised within an Oban task are not being caught by my error handler, even after registering it through Telemetry. I’m following along with the guide on the repo.

My handler (pretty much copied form docs):

defmodule MyApp.ErrorReporter do
  require Logger

  def handle_event([:oban, :job, :exception], measure, meta, _) do
    context =
      meta
      |> Map.take([:id, :args, :queue, :worker])
      |> Map.merge(measure)

    Logger.info("handling Oban exception")
    IO.inspect(meta, lablel: "meta")
    Honeybadger.notify(meta.error, context, meta.stack)
  end

  def handle_event([:oban, :circuit, :trip], _measure, meta, _) do
    context = Map.take(meta, [:name])

    Honeybadger.notify(meta.error, context, meta.stack)
  end
end

:ok = :telemetry.attach_many(
  "oban-errors",
  [[:oban, :job, :exception], [:oban, :circuit, :trip]],
  &Carriage.ErrorReporter.handle_event/4,
  nil
)

(Note that it’s a little odd that the attach_many call is happening here, instead of application.ex.. However, it does fire, as seen when wrapping it with IO.inspect()) and the docs have it in the error handler as well.

My Oban task:

defmodule MyApp.SendEmailAsync do
  use Oban.Worker, queue: :email, max_attempts: 1

  require Logger

  def perform(%{"param1" => param1}, _job) do
    Logger.info("the task is running!")
    raise "something went wrong"
  end

  def timeout(_job), do: :timer.seconds(60)
end

application.ex:

defmodule MyApp.Application do

  use Application
  
  def start(_type, _args) do

    ...

    :ok = Oban.Telemetry.attach_default_logger()

    ...

end

After running the server and executing the task, here’s what I see in logs:

[info] {"args":{"param1":"test"},"event":"started","queue":"email","source":"oban","start_time":1593026760599233,"worker":"MyApp.SendEmailAsync"}
[info] the task is running!
[info] {"args":{"param1":"test"},"duration":45834,"event":"failure","queue":"email","source":"oban","worker":"MyApp.SendEmailAsync"}

Output from the oban_jobs table:

 17 | discarded | email | MyApp.SendEmailAsync | {"param1": "test} | {"{\"at\": \"2020-06-24T19:26:00.602926Z\", \"error\": \"** (RuntimeError) something went wrong\\n    (my_app 0.2.0) lib/my_app/queues/email_sender.ex:20: MyApp.SendEmailAsync.perform/2\\n    (oban 1.2.0) lib/oban/queue/executor.ex:46: Oban.Queue.Executor.perform_inline/2\\n    (elixir 1.10.2) lib/task/supervised.ex:90: Task.Supervised.invoke_mfa/2\\n    (elixir 1.10.2) lib/task/supervised.ex:35: Task.Supervised.reply/5\\n    (stdlib 3.11) proc_lib.erl:249: :proc_lib.init_p_do_apply/3\\n\", \"attempt\": 1}"}                                                                |       1 |            1 | 2020-06-24 19:26:00.586236 | 2020-06-24 19:26:00.586236 | 2020-06-24 19:26:00.592488 | 2020-06-24 19:26:00.602913 | {JonsMacbookPro,email,jjvmltj4} |              |        0 | {}

It’s clear that the Telemetry logging is working as expected, the task runs, and that it hits the exception.

However, my ErrorReporter handle_event/4 functions were never called (I don’t see meta in logs, nor did the Honeybadger error get reported).

I might be missing something pretty simple. Any help appreciated!

Marked As Solved

jeroenvisser101

jeroenvisser101

Hey @jonbo, you might be using an older version. The README is for the unreleased v2.0.0-rc, you can see the implementation of the error telemetry event for v1.2 here:

https://github.com/sorentwo/oban/blob/v1.2.0/lib/oban/queue/executor.ex#L109

The documentation for the stable version can be found here: Oban — Oban v2.23.0

Hope that helps.

Also Liked

jonbo

jonbo

That was it, many thanks @jeroenvisser101!

For the record, I attached the logger with:

:telemetry.attach_many(
  "oban-errors",
  [[:oban, :failure]],
  &Carriage.ErrorReporter.handle_event/4,
  nil
)

And the handler:

  def handle_event([:oban, :failure], measure, meta, _) do
    context =
      meta
      |> Map.take([:id, :args, :queue, :worker])
      |> Map.merge(measure)

    IO.inspect(meta, lablel: "meta")
    Logger.info("handling Oban exception")
    Honeybadger.notify(meta.error, context, meta.stack)
  end
sorentwo

sorentwo

Oban Core Team

Thanks for solving this one @jeroenvisser101! This mismatch between the readme and the released version has been a persistent issue.

jeroenvisser101

jeroenvisser101

Not sure if it would help, but maybe adding a guides folder (like phoenix and others) and having the README just link to hexdocs would help?

Where Next?

Popular in Questions Top

_russellb
I want to try my hand at web scraping. What tools/libraries do I need to use. I’m hoping to turn this into something professional so don’...
New
JeremM34
Hello, how can I check the Phoenix version ? Thanks !
New
jononomo
I am trying to figure out how Mix knows whether the environment is test, dev, or prod – where is this set? Thanks.
New
shahryarjb
Hello, I get Persian date from my client and convert it to normal calendar like this: def jalali_string_to_miladi_english_number(persi...
New
pmjoe
I have a relationship of love and hate with Elixir. Lots of things are just absolutely right, but there are some things that are kind of ...
New
freewebwithme
Using vs code and installed ElixirLS: support and debugger. And I got an error popped up on start up says Failed to run ‘elixir’ comma...
New
JDanielMartinez
Hi! May someone helps me, please! I have two apps into an umbrella project: the first one is Database, which manages queries, and the se...
New
jason.o
In the code below, if the create action is not set to accept “extra_key” as an input, it errors out with a message shown above. Is there ...
New
chensan
I have a User schema with a :from_id field set to type :string: defmodule TweetBot.Repo.Migrations.CreateUsers do use Ecto.Migration ...
New
hariharasudhan94
Lets say I have map like this fetching from my database %{"_id" => #BSON.ObjectId<58eb1a7a9ad169198c3dXXXX>, "email" => ...
New

Other popular topics Top

danschultzer
None of the current solutions worked well for me, so I went ahead and built a user management system from scratch. This project took far...
548 29377 241
New
aadeshere1
I have a another noob question about loop. Since elixir is immutable, while loop is not directly possible. total = 10 while total != 0 ...
New
Harrisonl
We have an ECS cluster with 4 services, where each task joins a single cluster, via discovery ECS discovery service. Currently when I de...
New
dokuzbir
I want to highlight html closing tags when i click a html tag. That works in .html files but doesnt work for html.eex templates. How can...
New
SoCreat
i’m a new one to elixir which editor can i use vs code? or atom? Thanks! :smiley:
New
grych
Hi folks, Few months ago I have announced the proof-of-concept of the library to manipulate the browsers DOM objects directly from Elixi...
639 52341 488
New
ashish173
I am using Ecto timestamps with postgres, I can see the timestamps() use the :naive_dateime but for my use case I wanted to store the ti...
New
AstonJ
We’ve put together this wiki for Phoenix LiveView - please feel free to add any info you feel is worth including. What is Phoenix LiveV...
New
WestKeys
Currently suffering from paralysis by [HTTP client] analysis. This is rather unusual in Elixirland as there tends to be consensus on the ...
New
vonH
In asking this question I am more interested about the expressiveness of the language itself and less concerned about the availability of...
New

We're in Beta

About us Mission Statement