Telemetry metrics how to increment counter by a given amount?

Questions / Help
,
1

I think I don’t really get the idea of measurement options in Telemetry.Metrics.

I’ve got a function

def batch_process_jobs do
       # process jobs somewhere here...

        :telemetry.execute(
          [:my_app, :jobs],
          %{count: length(failed_task_ids)},
          %{status: "failed"}
        )

        :telemetry.execute(
          [:my_app, :jobs],
          %{count: length(succeeded_task_ids)},
          %{status: "succeeded"}
        )
end

So the idea is I want to see how many jobs failed and how many jobs succeeded after a batch process. My thinking is that if I give the number of failed and succeeded jobs (by calculating the length of IDs) as the count I’d expect that much increase in my bar graph as I am grouping by status.

Below you can see an example graph. As you can see both succeeded (light blue) and failed (dark blue) job count are the same however that’s not the case actually.

Screenshot 2022-01-24 at 16.22.30
Screenshot 2022-01-24 at 16.22.301129×501 5.83 KB

Why does that happen? When and how to use the measurement options like I tried in this case by giving count?

Edit: I figured that the count in the graph is actually the number of times :telemetry.execute was invoked for that specific metric type. Since they are in the same block, they are called the same number of times regardless of the count value I pass. Not sure whether that helps.

Cheers,
Burak

1 Like
2

I think I’ve figured it out, I’ve tested it as well so it worked.

We are using telemetry_metrics_statsd as the reporter and from its docs regarding the counter metric:

Note that the counter was bumped by 1, regardless of the measurements included in the event (careful reader will notice that the :count measurement we chose for the metric wasn’t present in the map of measurements at all!). Such behaviour conforms to the specification of counter as defined by Telemetry.Metrics package - a counter should be incremented by 1 every time a given event is dispatched.

What I should’ve used instead is the sum metric:

Sum metric is also represented as a gauge - the difference is that it always changes relatively and is never set to an absolute value. Given metric definition below

So I’ve changed this

counter("my_app.jobs.count", tags: [:status]),

into this:

sum("my_app.jobs.count",
  tags: [:status],
  reporter_options: [report_as: :counter]
)

I think I should’ve mentioned that in my question to start with, but I didn’t know it could be related. Sorry for that.

3 Likes