Looking for a bit of insight here to grow my skills around building observability into my elixir deployments.
I have a basic phoenix server and I am looking to publish statsd metrics to datadog via udp port 8125. I have a Telemetry module as produced by the phx bootstrapper without ecto that has TelemetryMetricsStatsd added to it:
Telemetry Module
defmodule StatsdDemoWeb.Telemetry do
use Supervisor
import Telemetry.Metrics
def start_link(arg) do
Supervisor.start_link(__MODULE__, arg, name: __MODULE__)
end
@impl true
def init(_arg) do
children = [
{TelemetryMetricsStatsd, metrics: metrics(), formatter: :datadog, port: 8125}
]
Supervisor.init(children, strategy: :one_for_one)
end
def metrics do
[
# Phoenix Metrics
summary("phoenix.endpoint.stop.duration",
unit: {:native, :millisecond}
),
summary("phoenix.router_dispatch.stop.duration",
tags: [:route],
unit: {:native, :millisecond}
),
counter("phoenix.socket_connected.count",
tags: [:endpoint]
),
# VM Metrics
summary("vm.memory.total", unit: {:byte, :kilobyte}),
summary("vm.total_run_queue_lengths.total"),
summary("vm.total_run_queue_lengths.cpu"),
summary("vm.total_run_queue_lengths.io")
]
end
end
When I run this natively in dev/prod environments (via iex -S mix phx.server) I can verify that the vm and phoenix metrics are emitted using netcat (nc -ulkp 8125).
However, when I run this inside of a docker container, it appears that I can no longer get the metrics emitted to the udp port. The docker compose image is the default produced by the mix release docker command:
Dockerfile
# Find eligible builder and runner images on Docker Hub. We use Ubuntu/Debian instead of
# Alpine to avoid DNS resolution issues in production.
#
# https://hub.docker.com/r/hexpm/elixir/tags?page=1&name=ubuntu
# https://hub.docker.com/_/ubuntu?tab=tags
#
#
# This file is based on these images:
#
# - https://hub.docker.com/r/hexpm/elixir/tags - for the build image
# - https://hub.docker.com/_/debian?tab=tags&page=1&name=bullseye-20220801-slim - for the release image
# - https://pkgs.org/ - resource for finding needed packages
# - Ex: hexpm/elixir:1.14.0-erlang-25.1-debian-bullseye-20210902-slim
#
ARG ELIXIR_VERSION=1.14.0
ARG OTP_VERSION=25.1
ARG DEBIAN_VERSION=bullseye-20220801-slim
ARG NODE_BULDER_IMAGE="node:18.3.0-alpine3.16"
ARG ELIXIR_BUILDER_IMAGE="hexpm/elixir:${ELIXIR_VERSION}-erlang-${OTP_VERSION}-debian-${DEBIAN_VERSION}"
ARG RUNNER_IMAGE="debian:${DEBIAN_VERSION}"
########################################
# 1. Node build stage
########################################
FROM ${NODE_BULDER_IMAGE} as node-builder
# prepare build dir
WORKDIR /app
RUN mkdir assets
# set build ENV
ENV NODE_ENV=prod
# install npm dependencies
COPY ./services/statsd_demo/assets/package.json ./assets/
COPY ./services/statsd_demo/assets/package-lock.json ./assets/
RUN npm install --prefix assets
########################################
# 2. Elixir build stage
########################################
FROM ${ELIXIR_BUILDER_IMAGE} as elixir-builder
# install build dependencies
RUN apt-get update -y && apt-get install -y build-essential git \
&& apt-get clean && rm -f /var/lib/apt/lists/*_*
# prepare build dir
WORKDIR /app
# install hex + rebar
RUN mix local.hex --force && \
mix local.rebar --force
# set build ENV
ENV MIX_ENV="prod"
# install mix dependencies
COPY ./services/statsd_demo/mix.exs ./
COPY ./services/statsd_demo/mix.lock ./
RUN mix deps.get --only $MIX_ENV
RUN mkdir config
# copy compile-time config files before we compile dependencies
# to ensure any relevant config change will trigger the dependencies
# to be re-compiled.
COPY ./services/statsd_demo/config/config.exs config/
COPY ./services/statsd_demo/config/${MIX_ENV}.exs config/
RUN mix deps.compile
COPY ./services/statsd_demo/priv priv
COPY ./services/statsd_demo/lib lib
COPY ./services/statsd_demo/assets assets
COPY --from=node-builder /app/assets/node_modules assets/node_modules
# compile assets
RUN mix assets.deploy
# Compile the release
RUN mix compile
# Changes to config/runtime.exs don't require recompiling the code
COPY ./services/statsd_demo/config/runtime.exs config/
COPY ./services/statsd_demo/rel rel
RUN mix release
########################################
# 2. Build run stage
#
# start a new build stage so that the final image will only contain
# the compiled release and other runtime necessities
########################################
FROM ${RUNNER_IMAGE}
RUN apt-get update -y && apt-get install -y libstdc++6 openssl libncurses5 locales \
&& apt-get clean && rm -f /var/lib/apt/lists/*_*
# Set the locale
RUN sed -i '/en_US.UTF-8/s/^# //g' /etc/locale.gen && locale-gen
ENV LANG en_US.UTF-8
ENV LANGUAGE en_US:en
ENV LC_ALL en_US.UTF-8
WORKDIR "/app"
RUN chown nobody /app
# set runner ENV
ENV MIX_ENV="prod"
# Only copy the final release from the build stage
COPY --from=elixir-builder --chown=nobody:root /app/_build/${MIX_ENV}/rel/socks ./
USER nobody
CMD ["/app/bin/server"]
I feel like I am missing something obvious, some declaration in the docker image or some configuration. Or perhaps it is working and I am testing it’s ability to emit from a docker container wrong.
Anyway, appreciate if someone could advise here or point out my lapse
