Sharing some ideas for a PromEx plugin that could be useful to Oban Pro users

Hi!

Just wanted to share some code that might be useful to Oban Pro users, it’s a simple PromEx custom plugin that exposes Oban.Met metrics to Prometheus.

PromEx already includes an Oban plugin but that one queries the database instead of requesting metrics from Oban.Met (which I believe it’s a Pro feature).

To reduce DB load I have experimented with some code to request metrics from Oban.Met rather than computing them again from the DB.

@sorenone @sorentwo any plans to release an official Oban Pro PromEx plugin? :smile:

defmodule ObanMetPromExPlugin do
  @moduledoc """
  Simple PromEx plugin that exposes `Oban.Met` stats to Prometheus.

  Right now it exposes a single metric called `oban_met_job_count` that
  counts jobs labelled with their queue and state.
  """

  use PromEx.Plugin

  @default_poll_rate 5_000
  @count_event_name [:oban, :met, :job, :count]

  @impl true
  def polling_metrics(opts) do
    poll_rate = Keyword.get(opts, :poll_rate, @default_poll_rate)

    Polling.build(
      :oban_met_job_count_metrics,
      poll_rate,
      {__MODULE__, :execute_queue_metrics, []},
      [
        last_value(
          [:oban, :met, :job, :count],
          event_name: @count_event_name,
          measurement: :count,
          description: "Oban.Met job count",
          tags: [:queue, :state]
        )
      ]
    )
  end

  def execute_queue_metrics() do
    queues =
      Oban.Met.checks(Oban)
      |> Enum.map(& &1["queue"])
      |> Enum.uniq()

    states = Enum.map(Oban.Job.states(), &Atom.to_string/1)

    for queue <- queues do
      states_counts =
        states
        |> Enum.into(%{}, fn state -> {state, 0} end)
        |> Map.merge(Oban.Met.latest(Oban, :full_count, group: "state", filters: [queue: queue]))

      for {state, count} <- states_counts do
        :telemetry.execute(@count_event_name, %{count: count}, %{queue: queue, state: state})
      end
    end
  end
end

1 Like

Thanks for sharing this, I hope some other people can make use of it as well.

The Oban.Met package is available with either Web or Pro.

There are some notes about it, but our backlog is deep :sunglasses:

Thanks for sharing your PromEx Pro Plugin @fedme! When I wrote the original Oban plugin for PromEx, I don’t think Oban.Met was around, so the plugin uses only what’s available from the open source Oban offering.

I wouldn’t be opposed to creating an Oban Pro derivative of the regular Oban plugin in PromEx…but at the moment I don’t have any bandwidth to take on that effort. If you think that this is something that you can tackle, I would be more than happy to provide guidance and get this into PromEx :).