Ignore Oban-related traces in OpenTelemetry for Ecto


I want to ignore Oban-related traces in Ecto OpenTelemetry which comes in two flavours:

Any advices how to approach this?

Me too. It is super annoying seeing millions of metrics being collected when the system is barely used.

Yesterday I turned open telemetry down for ecto because of this

We just went through this, it’s a bit involved. I’m on my phone so I’ll have to show code later but here is a high level answer:

  1. Attach a regular telemetry hook to ALL of the Oban internal events.
  2. In that hook, set an open telemetry context value of like “oban_internal: true”
  3. Configure your opentelemtry lib to use a custom sampler
  4. in your sampler, drop all traces where the trace context is Oban internal

As I said, a bit involved. I’ll be able to provide our code later.


All requests are made with Oban.Repo, which wraps calls with custom options as seen here: https://github.com/sorentwo/oban/blob/main/lib/oban/repo.ex#L97. Notice there is an oban: true field that you can detect in functions like prepare_query/3, as well as oban_conf in telemetry options which could theoretically be used to ignore Oban emitted events.

Unfortunately, it looks like OpenTelemetryEcto doesn’t have a way to ignore certain events (yet).

You could add a small wrapper module to your application, like this:

defmodule MyApp.OpenTelemetryEcto do
  @moduledoc """
  A wrapper around OpenTelemetryEcto that filters out queries generated by Oban.

  def setup(event_prefix, config \\ []) do
    event = event_prefix ++ [:query]

    :telemetry.attach({OpenTelemetryEcto, event}, event, &__MODULE__.handle_event/4, config)

  @doc false
  def handle_event(_event, _measure, %{oban_config: _}, _config) do

  def handle_event(event, measure, meta, config) do
    OpenTelemetryEcto.handle_event(event, measure, meta, config)

Interesting! We needed a sampler anyway but that does look like an easier way to do it! Will test out later today.

After playing with this locally this clause is slightly wrong, it needs to be:

  # filter out oban tagged values
  def handle_event(event, measure, %{options: options} = meta, config) do
    if Keyword.has_key?(options, :oban_conf) do
      OpentelemetryEcto.handle_event(event, measure, meta, config)
1 Like

Good to know! Sorry, I wrote that without checking that it worked because I don’t currently use OpenTelemetry :slightly_smiling_face:

1 Like