How to test OpenTelemetry(API)?

In the application level, I think all we need is to test that OpenTelemetry’s functions are being called, rather than test how they perform and what they produce.

With Mox it might be something like

# define mock in helper
Mox.defmock(OpenTelemetry.SpanMock, for: OpenTelemetry.Span)

# in test case
expect(OpenTelemetry.SpanMock, :set_attributes, fn [result: :world] -> :ok end)
MyModule.hello()

With this we are making sure that MyModule.hello() invokes OpenTelemetry’s set_attributes function with our desired attribute. (Similarly we can write an expectation for Tracer.with_span).

For the rest, (wether it’s recorded, sampled) I would rely on tests in library and assume that it works properly.

In other words, in my application I just make sure that I gave 3rd party lib required info - the rest is not my responsibility

Yeah, I think that’s good advice for client apps. I didn’t know Span defined a behavior and hadn’t thought about mocking the set_attributes method, so that’s good to know!

I’m actually working on a library that defines and links spans so it’s nice to have a bit more control. A couple folks on slack pointed me toward the pid and ets exporters. The example they linked registers the pid exporter to send spans to the test process and then assert that the received span has the correct attributes.

https://github.com/opentelemetry-beam/opentelemetry_phoenix/blob/main/test/opentelemetry_phoenix_test.exs#L99-L166

setup do
    :application.stop(:opentelemetry)
    :application.set_env(:opentelemetry, :tracer, :ot_tracer_default)

    :application.set_env(:opentelemetry, :processors, [
      {:ot_batch_processor, %{scheduled_delay_ms: 1}}
    ])

    :application.start(:opentelemetry)

    :ot_batch_processor.set_exporter(:ot_exporter_pid, self())
    :ok
  end

  test "records spans for Phoenix web requests" do
    OpentelemetryPhoenix.setup()

    :telemetry.execute(
      [:phoenix, :endpoint, :start],
      %{system_time: System.system_time()},
      %{conn: @conn, options: []}
    )

    :telemetry.execute(
      [:phoenix, :router_dispatch, :start],
      %{system_time: System.system_time()},
      %{
        conn: @conn,
        plug: MyStoreWeb.UserOrdersController,
        plug_opts: :index,
        route: "/users/{user_id}/orders",
        path_params: %{"user_id" => "123"}
      }
    )

    :telemetry.execute(
      [:phoenix, :endpoint, :stop],
      %{duration: 444},
      %{conn: stop_conn(@conn), options: []}
    )

    expected_status = OpenTelemetry.status(:Ok, "Ok")

    assert_receive {:span,
                    span(
                      name: "GET /users/{user_id}/orders",
                      attributes: list,
                      status: ^expected_status
                    )}

    assert [
             {"http.client_ip", "10.211.55.2"},
             {"http.host", "mystore"},
             {"http.method", "GET"},
             {"http.scheme", "http"},
             {"http.status", 200},
             {"http.target", "/users/123/orders"},
             {"http.user_agent",
              "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:79.0) Gecko/20100101 Firefox/79.0"},
             {"net.host.ip", "10.211.55.2"},
             {"net.host.port", 4000},
             {"net.peer.ip", "10.211.55.2"},
             {"net.peer.port", 64291},
             {"net.transport", "IP.TCP"},
             {"phoenix.action", "index"},
             {"phoenix.plug", "Elixir.MyStoreWeb.UserOrdersController"}
           ] == List.keysort(list, 0)
  end

Nice!

Honestly, I haven’t worked with OpenTelemetry, and don’t know if OpenTelemetry.Span defines behaviour (seems like it’s not)
But for mocking with Mox if behaviour is not defined by lib I would write it and have the lib’s original module as one of the implementations.

Thank you for following up in here with those ideas!