OpenCensus gains new integrations with Elixir libraries

Using trace data is likely a bad idea. There can only be one tracer at any point in time (no concurrency), so if you integrate it to the metrics, nobody can use it to debug at the same time.

Yeah, I have checked that and it is a shame. So I would need to find another way to handle that, unfortunately I cannot take a look into WombatOAM

By the way, new exporter (not yet published) is available:

• InfluxDB

I’m happy to see that OpenCensus is gaining more traction in the BEAM ecosystem! I think it was about 6-9 months ago that I last checked and it didn’t seem like there was enough there yet to go all in, but now it seems that there are a number of people working on it.

I am maintaining the Spandex project along with @zachdaniel, with the goal of being an OpenTracing-compatible API that supports multiple vendor back-ends, very similar to the goals of OpenCensus. Currently, we only support DataDog APM as a back-end, but I think it should be relatively simple to support others if someone wanted to (and I’d love to know where they got stuck if it didn’t work out). We’re also looking forward to using Telemetry to collect span data in the future as more libraries support it. In the meantime, you may be interested to look at how Spandex integrates with Plug/Phoenix and Ecto. We have an example project that demonstrates how to integrate Spandex into an existing app, if you’re curious (see the links in the README for a code diff).

Also, as you probably know, several vendors have their own first-party support for tracing that is proprietary to only their back-ends, but mostly open-source (e.g. AppSignal, New Relic, Scout).

I’d be interested to look more into how the OpenCensus BEAM core compares to Spandex, and see whether there’s anything that can be shared somehow so everybody wins.

I agree :). I spent a really long time on Spandex, but I don’t want to be working against a more standardized implementation of tracing in Elixir. I’d hope that either spandex could be a standard interface for tracing an Elixir application, or the good parts of it could be salvaged/copied into a new standard. My point being that we should only hold tight into spandex if it’s going to be good for the community. If the community needs a more standard implementation, I’d be happy to support it.

I see no problems with making Spandex thin wrapper over OpenCensus library as API and concepts are quite similar. To be honest I even thought about creating OpenCensus wrapper for Elixir, that would end like something like Spandex, but in the end I didn’t see much value in that (at least yet, because I have proposed moving some utilities to macros). I also believe that we should reduce fragmentation in the matter of tracing and metrics gathering, but not only in Elixir, but in BEAM as a platform. It would be best if we would have single toolset to export metrics form both Erlang and Elixir.

Main reasons why I have joined OpenCensus project rather than Spandex are:

• OpenCensus is Erlang project, so by adding new integrations there I can improve all BEAM languages at once. While we can still use Elixir from other languages in BEAM-family it isn’t that pretty, especially as a lot of Elixir code relies on Elixir’s macros. Porting Erlang macro to Elixir is much more straightforward than other way around.
• In theory OpenCensus have bigger backing by being Google project that was open-sourced. It also has some big names in “Partners & Contributors” section.
• OpenCensus also provide interface for metrics gathering, and I wanted something for my project Imager that would provide me single interface to export metrics and traces for different systems defined at runtime (PR with support for that is still WIP).
• It uses W3C Context Specification for distributed tracing instead of relying on DataDog proprietary headers.

My only main problem with OpenCensus library right now is that it completely lacks documentation, which I will try to tackle soon. A lot of data is outdated, and it would be really wort to improve that, as it could help building community around such standard.

About integrating with Plug/Phoenix and Ecto I would need to create example projects in opencensus/plug and opencensus/telemetry. Only place where I still need to find how to hook into Ecto is tracing, but I have seen that you are using :trace level of Ecto logger for that, it can be worth checking this out that for adding new integration.

Another useful project would be some benchmarks to both check overhead and eventually compare different ways of setting traces and collecting metrics. For example, I have a work in progress for using a message to a pool of workers for collecting metrics instead of doing them inline. In many cases just doing them inline is likely preferred – when not updating lots of views per measure – but useful to have numbers.

I should also mention that there are other OpenTracing-related projects in the BEAM ecosystem that seem relatively robust as options: otter for Erlang project and ExRay, which is an Elixir wrapper around it.

I agree that the ideal situation would be a robust core that allows the difficult/complicated parts to be shared by all BEAM languages, and perhaps wrapper interfaces native to each language so that it feels more idiomatic (for example, with respect to the macros). I think Spandex could be adapted somewhat-easily to that because it doesn’t rely in any magic or macros at its core - only for convenience so that we can offer function decorators and automatically generate some code for you so that you don’t have to explicitly pass in all of the default options to each API call.

I was fully aware of other solutions available out there. The points that (at least for me) were the most important are:

• OpenCensus Erlang library seems to be backed by Google, and even if it isn’t then it seems more “official” than community projects like otter and Spandex
• It supports not only tracing but metrics as well. Having common API for both of these is quite handy.
• Is supported by the author of the Prometheus implementation which I quite liked and used in some projects.
• Adding new integrations was a breeze
• Metrics supports tags (in contrast to exometer/elixometer or Folsom)

So everything I needed was in just one package, what I really appreciate. So, with all respect to you guys, I place my bet on OpenCensus.

Yea, I would love if we could all share an Erlang core. But now that there are so many implementations I doubt that is ever going to happen. And I’m partly to blame :). I started opencensus-erlang after trying otter when it was released.

I find opencensus to be important as a core because it defines so much (like propagation encoding and even a stacktrace proto format so services will actually be able to understand Erlang stacktraces! – small potatoes to Elixir world since services are starting to pick up on supporting Elixir).

It is also very open development of the specs and accepted Erlang as part of this process from early on.

opencensus_plug has landed on Hex