What tools do you use for logging / metrics?

Logging

Like fluentd or maybe something else? Where do you store the logs? I’m mostly interested in “self-hosted” (i.e. not saas) solutions.

Metrics

Prometheus or maybe something native to erlang? How do you manage prometheus nodes?

3 Likes

For metrics, I use exometer to gather erlang stats and some custom metrics, which are sent to statsd, and from there to datadog(SaaS), where I can visualize them. I’ve been quite happy with this setup but would still be interested to hear about other solutions in use around the forum.

2 Likes

We use Exometer for metrics, and Prometheus to collect them. We wrote this module to format metrics: https://github.com/cogini/prometheus_exometer

We host our own Prometheus instances, provisioned with Ansible. We use Grafana for graphs.

We use Elasticsearch / Logstash / Kibana for logs. We have found AWS Elasticsearch to be flaky and slow to release updated versions, so we generally run our own instances even in cloud hosted applications.

We send transaction logs via Kafka directly from Elixir, then do real time stream processing on them in Elixir and other tools. We are using brod directly, rather than one of the Elixir wrappers.

5 Likes

We use statix library sending data to telegraf… the results end up in Influx and Chronograf makes it super easy to visualize the metrics.

1 Like

currently it’s wrapped opencensus plus elasticsearch for logs and prometheus for metrics.

1 Like