So yes benchee is more for measuring performance before something goes live or for testing out multiple approaches locally, from what I understand you want to have more of an introspection into the running system. @jola offered multiple great solutions there.
Adopting Elixir, which is great, has a chapter about Metrics and instrumenting your system.
Other projects they mention as being used in the wild:
- exometer for pushing performance data to services such as Data Dog
- football addicts has their own write up of how they use Telegraf, InfluxDB and Grafana
- Prometheus and Grafana is yet another choice
I like whole Application Performance Monitoring Solutions in the SaaS space such as:
- AppSignal
- Scout
- wombatoam
- there are some community developed agents for NewRelic, not sure how good they are 1 2
And for even more options you could take a look at some of our previous threads on the subject: