OpenTelemetry exporter failed to initialize with exception inets_not_started

Questions / Help
, , , ,
1

I’m trying to deploy Phoenix application with OpenTelemetry tracing in a Docker container. Dockerfile I was using so far (sucessfully) is a typical one (i.e. generated by mix phx.gen.release --docker). Following versions are used:

ARG ELIXIR_VERSION=1.15.4
ARG OTP_VERSION=26.0.2
ARG DEBIAN_VERSION=bullseye-20230612-slim

So, the image is hexpm/elixir:1.15.4-erlang-26.0.2-debian-bullseye-20230612-slim.
(I’m running the same versions locally for development, the only difference I’m working on Ubuntu 22.04 via WSL2.)

The problem is that then running this container, a following message is produced which indicates that OTLP exporter fails (with MIX_ENV=prod):

=WARNING REPORT==== 6-Sep-2023::20:15:42.868641 ===
OTLP exporter failed to initialize with exception error:{badmatch,
                                                          {error,
                                                           inets_not_started}}
=WARNING REPORT==== 6-Sep-2023::20:15:42.880076 ===
Failed to load OTP-trusted CAs: error:{badmatch,{error,enoent}}, falling back to hardcoded authorities

This doesn’t happen when running code in (both in dev and prod) locally, outside container.

My OTLP config is almost the same as in Getting Started | OpenTelemetry, namely:

mix.exs:

# (in this order):

{:opentelemetry_exporter, "~> 1.6"},
{:opentelemetry, "~> 1.3"},
{:opentelemetry_api, "~> 1.2"},
{:opentelemetry_phoenix, "~> 1.1"},
{:opentelemetry_cowboy, "~> 0.2"},
{:opentelemetry_ecto, "~> 1.1"},

# and:

defp releases() do
  [
    foo: [
      applications: [
        opentelemetry: :temporary,
        opentelemetry_exporter: :permanent
      ]
    ]
  ]
end

application.ex:

:opentelemetry_cowboy.setup()
OpentelemetryPhoenix.setup(adapter: :cowboy2)
OpentelemetryEcto.setup([:foo, :repo])

config/runtime.ex:

  otlp_protocol = System.get_env("OTEL_PROTOCOL") |> String.to_atom()
  otlp_endpoint = System.get_env("OTEL_ENDPOINT")

  config :opentelemetry,
    span_processor: :batch,
    traces_exporter: :otlp

  config :opentelemetry_exporter,
    otlp_protocol: otlp_protocol,
    otlp_endpoint: otlp_endpoint

  config :opentelemetry, :processors, [
    {:otel_simple_processor, %{}}
  ]

Does anyone have any idea why this problem occurs?

Edit:

I’ve figured out that the problem only occurs in a release, that’s why I only see it in a container, not when starting via mix phx.server or iex -S mix phx.server.

Edit 2:

When run with lower log level I also get:

[debug] OTLP exporter failed to initialize: exception error: no match of right hand side value {error,inets_not_started}
  in function  opentelemetry_exporter:start_httpc/1 (/home/poly/P/foo-api/deps/opentelemetry_exporter/src/opentelemetry_exporter.erl, line 239)
  in call from opentelemetry_exporter:init/1 (/home/poly/P/foo-api/deps/opentelemetry_exporter/src/opentelemetry_exporter.erl, line 210)
  in call from otel_exporter:init/1 (/home/poly/P/foo-api/deps/opentelemetry/src/otel_exporter.erl, line 44)
  in call from otel_batch_processor:init_exporter/2 (/home/poly/P/foo-api/deps/opentelemetry/src/otel_batch_processor.erl, line 317)
  in call from otel_batch_processor:idle/3 (/home/poly/P/foo-api/deps/opentelemetry/src/otel_batch_processor.erl, line 197)
  in call from gen_statem:loop_state_callback/11 (gen_statem.erl, line 1377)
  in call from proc_lib:init_p_do_apply/3 (proc_lib.erl, line 241)
[warning] OTLP exporter failed to initialize with exception :error:{:badmatch, {:error, :inets_not_started}}

Thanks!

2

It turned out that changing the order of opentelemetry and opentelemetry_exporter under releases in mix.exs fixes the issue:

foo: [
  applications: [
    opentelemetry_exporter: :permanent,
    opentelemetry: :temporary
  ]
]
3 Likes