neenjaw

neenjaw

Looking for a bit of insight here to grow my skills around building observability into my elixir deployments.

I have a basic phoenix server and I am looking to publish statsd metrics to datadog via udp port 8125. I have a Telemetry module as produced by the phx bootstrapper without ecto that has TelemetryMetricsStatsd added to it:

Telemetry Module
defmodule StatsdDemoWeb.Telemetry do
  use Supervisor
  import Telemetry.Metrics

  def start_link(arg) do
    Supervisor.start_link(__MODULE__, arg, name: __MODULE__)
  end

  @impl true
  def init(_arg) do
    children = [
      {TelemetryMetricsStatsd, metrics: metrics(), formatter: :datadog, port: 8125}
    ]

    Supervisor.init(children, strategy: :one_for_one)
  end

  def metrics do
    [
      # Phoenix Metrics
      summary("phoenix.endpoint.stop.duration",
        unit: {:native, :millisecond}
      ),
      summary("phoenix.router_dispatch.stop.duration",
        tags: [:route],
        unit: {:native, :millisecond}
      ),
      counter("phoenix.socket_connected.count",
        tags: [:endpoint]
      ),

      # VM Metrics
      summary("vm.memory.total", unit: {:byte, :kilobyte}),
      summary("vm.total_run_queue_lengths.total"),
      summary("vm.total_run_queue_lengths.cpu"),
      summary("vm.total_run_queue_lengths.io")
    ]
  end
end

When I run this natively in dev/prod environments (via iex -S mix phx.server) I can verify that the vm and phoenix metrics are emitted using netcat (nc -ulkp 8125).

However, when I run this inside of a docker container, it appears that I can no longer get the metrics emitted to the udp port. The docker compose image is the default produced by the mix release docker command:

Dockerfile
# Find eligible builder and runner images on Docker Hub. We use Ubuntu/Debian instead of
# Alpine to avoid DNS resolution issues in production.
#
# https://hub.docker.com/r/hexpm/elixir/tags?page=1&name=ubuntu
# https://hub.docker.com/_/ubuntu?tab=tags
#
#
# This file is based on these images:
#
#   - https://hub.docker.com/r/hexpm/elixir/tags - for the build image
#   - https://hub.docker.com/_/debian?tab=tags&page=1&name=bullseye-20220801-slim - for the release image
#   - https://pkgs.org/ - resource for finding needed packages
#   - Ex: hexpm/elixir:1.14.0-erlang-25.1-debian-bullseye-20210902-slim
#
ARG ELIXIR_VERSION=1.14.0
ARG OTP_VERSION=25.1
ARG DEBIAN_VERSION=bullseye-20220801-slim

ARG NODE_BULDER_IMAGE="node:18.3.0-alpine3.16"
ARG ELIXIR_BUILDER_IMAGE="hexpm/elixir:${ELIXIR_VERSION}-erlang-${OTP_VERSION}-debian-${DEBIAN_VERSION}"
ARG RUNNER_IMAGE="debian:${DEBIAN_VERSION}"

########################################
# 1. Node build stage
########################################
FROM ${NODE_BULDER_IMAGE} as node-builder

# prepare build dir
WORKDIR /app
RUN mkdir assets

# set build ENV
ENV NODE_ENV=prod

# install npm dependencies
COPY ./services/statsd_demo/assets/package.json ./assets/
COPY ./services/statsd_demo/assets/package-lock.json ./assets/
RUN npm install --prefix assets

########################################
# 2. Elixir build stage
########################################
FROM ${ELIXIR_BUILDER_IMAGE} as elixir-builder

# install build dependencies
RUN apt-get update -y && apt-get install -y build-essential git \
  && apt-get clean && rm -f /var/lib/apt/lists/*_*

# prepare build dir
WORKDIR /app

# install hex + rebar
RUN mix local.hex --force && \
  mix local.rebar --force

# set build ENV
ENV MIX_ENV="prod"

# install mix dependencies
COPY ./services/statsd_demo/mix.exs ./
COPY ./services/statsd_demo/mix.lock ./
RUN mix deps.get --only $MIX_ENV
RUN mkdir config

# copy compile-time config files before we compile dependencies
# to ensure any relevant config change will trigger the dependencies
# to be re-compiled.
COPY ./services/statsd_demo/config/config.exs config/
COPY ./services/statsd_demo/config/${MIX_ENV}.exs config/
RUN mix deps.compile

COPY ./services/statsd_demo/priv priv

COPY ./services/statsd_demo/lib lib

COPY ./services/statsd_demo/assets assets
COPY --from=node-builder /app/assets/node_modules assets/node_modules

# compile assets
RUN mix assets.deploy

# Compile the release
RUN mix compile

# Changes to config/runtime.exs don't require recompiling the code
COPY ./services/statsd_demo/config/runtime.exs config/

COPY ./services/statsd_demo/rel rel
RUN mix release

########################################
# 2. Build run stage
#
# start a new build stage so that the final image will only contain
# the compiled release and other runtime necessities
########################################
FROM ${RUNNER_IMAGE}

RUN apt-get update -y && apt-get install -y libstdc++6 openssl libncurses5 locales \
  && apt-get clean && rm -f /var/lib/apt/lists/*_*

# Set the locale
RUN sed -i '/en_US.UTF-8/s/^# //g' /etc/locale.gen && locale-gen

ENV LANG en_US.UTF-8
ENV LANGUAGE en_US:en
ENV LC_ALL en_US.UTF-8

WORKDIR "/app"
RUN chown nobody /app

# set runner ENV
ENV MIX_ENV="prod"

# Only copy the final release from the build stage
COPY --from=elixir-builder --chown=nobody:root /app/_build/${MIX_ENV}/rel/socks ./

USER nobody

CMD ["/app/bin/server"]

I feel like I am missing something obvious, some declaration in the docker image or some configuration. Or perhaps it is working and I am testing it’s ability to emit from a docker container wrong.

Anyway, appreciate if someone could advise here or point out my lapse

Showing Posts 1 to 1

neenjaw

neenjaw OP

I was modelling this off of another service (non-elixir) that I am intending to replace. Turns out there were obscure kubernetes configuration overlays injecting ENV_VARS to direct stats to a different host implicitly.

TL;DR, statsd needs to have a host outside of local host configured for the statsd packet to be sent properly.

— All posts loaded —

Where Next? Top

Trending in Questions Top

stjefim
Hello! Suppose you are building workflow (order / task / payment) processing system with the following requirements: Each workflow con...
New
water
I’m aware of 2 such projects, are there others? git: GitHub - team-alembic/realworld: A fullstack Phoenix LiveView application with back...
New
peck
Hi everyone. My team and I have been working on a fairly modest app based around video streaming and chat, but we’ve landed a customer t...
New
bradley
I really like the adapter patterns that ecto, nebulex, waffle, etc. use and would love find something similar for a key management servic...
New
unaware8150
Hello folks! So at work, we are seeing some situations where we have to define some “fixed” strings that are used across the codebase in...
New
Alvinkariuki
How Can I Optimise Compile Time Dependencies I have been building an elixir application for about 2 years now. Many modules and files ha...
New
leoschweizer
Hi, we run different LLM models across a cluster, with each node loading a configurable subset. Model-specific queues let nodes consume ...
New

Other Trending Topics Top

mudasobwa
I fully migrated to my own harness from Anthropic/Gemini and I think it’s time to share it. Welcome DSH, the DeepSeek Harness, fully writ...
New
KristerV
Hey. Is there anyone here who creates agents in their apps? Not talking about using agents, but creating them. I’m finding it pretty diff...
New
jimsynz
Beam Bots (or just BB for short) is a framework for building fault-tolerant robotics applications in Elixir using familiar OTP patterns. ...
New
juhalehtonen
There has been a thread to discuss the Stack Overflow Developer Survey on this forum every year since 2018, so here’s yet another one for...
New
webofbits
With AI doing more of the implementation work, I’ve been wondering how much coding I should deliberately keep doing myself. My main conc...
#ai
New
budgie
I love Elixir. It’s one of 2 programming languages I’ve ever fallen in love with. But I don’t use it anymore. Serverless was the promis...
New

We're in Beta

About us Mission Statement

Options

Thread Display Mode




Thread Preview

Skip Thread Previews