jonbo

jonbo

Oban/Telemetry error handling not capturing exceptions

Hi all,

Trying to figure out exceptions raised within an Oban task are not being caught by my error handler, even after registering it through Telemetry. I’m following along with the guide on the repo.

My handler (pretty much copied form docs):

defmodule MyApp.ErrorReporter do
  require Logger

  def handle_event([:oban, :job, :exception], measure, meta, _) do
    context =
      meta
      |> Map.take([:id, :args, :queue, :worker])
      |> Map.merge(measure)

    Logger.info("handling Oban exception")
    IO.inspect(meta, lablel: "meta")
    Honeybadger.notify(meta.error, context, meta.stack)
  end

  def handle_event([:oban, :circuit, :trip], _measure, meta, _) do
    context = Map.take(meta, [:name])

    Honeybadger.notify(meta.error, context, meta.stack)
  end
end

:ok = :telemetry.attach_many(
  "oban-errors",
  [[:oban, :job, :exception], [:oban, :circuit, :trip]],
  &Carriage.ErrorReporter.handle_event/4,
  nil
)

(Note that it’s a little odd that the attach_many call is happening here, instead of application.ex.. However, it does fire, as seen when wrapping it with IO.inspect()) and the docs have it in the error handler as well.

My Oban task:

defmodule MyApp.SendEmailAsync do
  use Oban.Worker, queue: :email, max_attempts: 1

  require Logger

  def perform(%{"param1" => param1}, _job) do
    Logger.info("the task is running!")
    raise "something went wrong"
  end

  def timeout(_job), do: :timer.seconds(60)
end

application.ex:

defmodule MyApp.Application do

  use Application
  
  def start(_type, _args) do

    ...

    :ok = Oban.Telemetry.attach_default_logger()

    ...

end

After running the server and executing the task, here’s what I see in logs:

[info] {"args":{"param1":"test"},"event":"started","queue":"email","source":"oban","start_time":1593026760599233,"worker":"MyApp.SendEmailAsync"}
[info] the task is running!
[info] {"args":{"param1":"test"},"duration":45834,"event":"failure","queue":"email","source":"oban","worker":"MyApp.SendEmailAsync"}

Output from the oban_jobs table:

 17 | discarded | email | MyApp.SendEmailAsync | {"param1": "test} | {"{\"at\": \"2020-06-24T19:26:00.602926Z\", \"error\": \"** (RuntimeError) something went wrong\\n    (my_app 0.2.0) lib/my_app/queues/email_sender.ex:20: MyApp.SendEmailAsync.perform/2\\n    (oban 1.2.0) lib/oban/queue/executor.ex:46: Oban.Queue.Executor.perform_inline/2\\n    (elixir 1.10.2) lib/task/supervised.ex:90: Task.Supervised.invoke_mfa/2\\n    (elixir 1.10.2) lib/task/supervised.ex:35: Task.Supervised.reply/5\\n    (stdlib 3.11) proc_lib.erl:249: :proc_lib.init_p_do_apply/3\\n\", \"attempt\": 1}"}                                                                |       1 |            1 | 2020-06-24 19:26:00.586236 | 2020-06-24 19:26:00.586236 | 2020-06-24 19:26:00.592488 | 2020-06-24 19:26:00.602913 | {JonsMacbookPro,email,jjvmltj4} |              |        0 | {}

It’s clear that the Telemetry logging is working as expected, the task runs, and that it hits the exception.

However, my ErrorReporter handle_event/4 functions were never called (I don’t see meta in logs, nor did the Honeybadger error get reported).

I might be missing something pretty simple. Any help appreciated!

Marked As Solved

jeroenvisser101

jeroenvisser101

Hey @jonbo, you might be using an older version. The README is for the unreleased v2.0.0-rc, you can see the implementation of the error telemetry event for v1.2 here:

https://github.com/sorentwo/oban/blob/v1.2.0/lib/oban/queue/executor.ex#L109

The documentation for the stable version can be found here: Oban — Oban v2.23.0

Hope that helps.

Also Liked

jonbo

jonbo

That was it, many thanks @jeroenvisser101!

For the record, I attached the logger with:

:telemetry.attach_many(
  "oban-errors",
  [[:oban, :failure]],
  &Carriage.ErrorReporter.handle_event/4,
  nil
)

And the handler:

  def handle_event([:oban, :failure], measure, meta, _) do
    context =
      meta
      |> Map.take([:id, :args, :queue, :worker])
      |> Map.merge(measure)

    IO.inspect(meta, lablel: "meta")
    Logger.info("handling Oban exception")
    Honeybadger.notify(meta.error, context, meta.stack)
  end
sorentwo

sorentwo

Oban Core Team

Thanks for solving this one @jeroenvisser101! This mismatch between the readme and the released version has been a persistent issue.

jeroenvisser101

jeroenvisser101

Not sure if it would help, but maybe adding a guides folder (like phoenix and others) and having the README just link to hexdocs would help?

Where Next?

Popular in Questions Top

Harrisonl
We have an ECS cluster with 4 services, where each task joins a single cluster, via discovery ECS discovery service. Currently when I de...
New
ovidiubadita
Hey all, I discovered Elixir and I love it. I always wanted to learn a functional programming and I intended to go for Haskell, but afte...
New
vac
Hi, I’m quite new in Elixir and I’m trying to format a string to a PEM format. I have the certificate value like MIIDBTCCAe2...... and I...
New
jay1
Why is it that the mnesia database isn’t the most preferred database for use in Elixir/Phoenix?
New
Emily
I have VueJS GUIs with the project generated using Webpack. I have Elixir modules that will need to be used by the VueJS GUIs. I forese...
New
jason.o
In the code below, if the create action is not set to accept “extra_key” as an input, it errors out with a message shown above. Is there ...
New
romenigld
I am trying to run a deploy with docker and I successfully runned with this command: docker build -t romenigld/blog-prod . but when I t...
New
komlanvi
Hi everyone, I was playing with phoenix liveView but I run into an issue. I have a form and want to validate each input text when the te...
New
shijith.k
I am trying to start a new phoenix project with elixir 1.9, but mix phx.new does not work. It says that ** (Mix) The task "phx.new" could...
New
svb
Hi! Currently I want to submit a form by pressing the Enter key. However, since my input field is of type “textarea” this is just adds a...
New

Other popular topics Top

senggen
Erlang/OTP 25 [erts-13.2.2] [source] [64-bit] [smp:8:8] [ds:8:8:10] [async-threads:1] 15:22:35.803 [error] gen_event {lager_file_backend...
New
lastday4you
I wanted to check elixir version in phoenix because i found that my elixir is 1.5 but when i use Enum.chunk_by it said the function is un...
New
TunkShif
This post is an instruction guide to help you setup your Neovim for Elixir development from scratch. It includes general information on h...
274 41539 114
New
Nvim
Anybody knows a comprehensive comparison of Django and Phoenix, thanks for the help. Where are they similar? Where do they differ the m...
New
stefanchrobot
What’s the safe way to decode a JSON string into a struct? I want to avoid calling String.to_atom. Jason.decode can give me a map with st...
New
Lily
In templates/appointment/index.html.eex: <%= for appointment <- @appointments do %> <tr> <td><%= appoi...
New
joeerl
Hello again - after a longish gap I’ve decided I really must dig into Elixir and see what’s been happening here - so I have a few questio...
New
aalberti333
As the title describes, I’m trying to run Enum.map() over a list of key/value pairs, where the value is a map. My data looks like this: ...
New
fayddelight
I tried installing elixir 1.11.2 erlang 23.3.4 via asdf in my zsh shell. Enabled the versions locally and globally. When I list them ...
New
jason.o
In the code below, if the create action is not set to accept “extra_key” as an input, it errors out with a message shown above. Is there ...
New

We're in Beta

About us Mission Statement