Afaik timber is in lowest level of support mode by their creators, while they’re working on https://vector.dev/. Personally I’ve moved away from them to AppSignal, as I actually needed exception tracking, while logs are fine stored locally by systemd. There are many other tools out there doing log aggregation, but not all of them might have specific elixir integrations (and depending on your sysadmin skills might not even need one)
Yeah I’m a paying customer but have yet to ever hear back from their support. As soon as I get the time we will move our product to another service.
btw our stack traces look like this (looks like a new bug I have to fix), and you can easily bring up all of them with the query:
We have had very similar experience, and preemptively moved everything to AppSignal. It’s a shame because they had very good Elixir exception catching library. Only now, with 2.0-beta AppSignal is catching up with it, and will be able to actually catch all exceptions in the system.
But anyway, not all start-ups succeed, in fact most don’t.
No. But they continue to charge (even emailing me asking to upgrade my service) but provide flakey service and zero customer support. It’s not a great way of operating.
I use AppSignal for error reporting & instrumentation and I’m very happy with it.
Now I need to find a logging aggregator that is stable, production ready and easy to use.
@chasers would you suggest Logflare right now for my use-case?
Yep, we are stable and production ready. I’ve been using our own backend to push our prod Elixir logs to staging for qa. I also use our Cloudflare app to push CDN logs to prod. So I very much use Logflare for Logflare.
Would love to have you on board, and I’d be happy to jump on a demo or something if you like. I feel like our Elixir logger stuff is fairly self explanatory but if you need help just let me know!
edit: FWIW we have over 2,000 active “sources” sending data daily, and a number of them are sending quite a lot of data.
@jesse The Gigalixir “drains” UI has 2 placeholder “example” URLs for drains. One is papertrail and the other is timber. Can you maybe remove the timber one, since timber seems to be dead? Maybe a shiny new logflare URL instead? e.g. https://api.logflare.app/logs/syslog?api_key=API_KEY&source=SOURCE_ID
here is the current UI:
Would love to swap that link out
I’ve also had Timber support be non-responsive and am evaluating alternatives for when their libraries are abandoned. It’s a shame because I love quite a few things about their platform:
- Ease-of-use and great UI.
- Multiple “tabs” of logs with easy search
- Great Elixir libraries. They were well documented and easy to work with.
Curious which libs from them are you using?
We use DataDog at my company, I’ve generally been quite happy with their logging solution, and the ability to switch back and forth between logs and metrics within the same timespan has been quite helpful.
Oh yes! I’ll do that. I want to test it out to make sure it all works correctly first, but we’ll try and do that very soon. Thanks for letting me know about this.
@chasers Thanks for making Logflare! It looks pretty great. I tried out the URL @slouchpie provided, but ran into a small problem. Logflare receives the data and displays what it received, but does not seem to parse the lines according to the format we are sending. I looked around for documentation, but couldn’t find anything that exactly fits.
We drain logs to HTTPS endpoints using the same format as Heroku, described here: https://devcenter.heroku.com/articles/log-drains#https-drains. In fact, we use Heroku’s logplex router to do the draining so supporting this format means you’ll support both Heroku and Gigalixir.
This is an example log line I see on Logflare.
217 <190>1 2020-08-08T16:12:15.382251+00:00 host bar bar-587b6c56bd-6j9sq - e[36mweb.1 | e[0m16:12:15.381 [warn] [libcluster:k8s_example] kubernetes strategy is selected, but :kubernetes_node_basename is not configured!
217 <190>1, which is metadata.
Also, as a side note, would you be able to handle the ANSI control sequences, such as those surrounding
web.1 | inside the logs, which change the colors? For what it’s worth, Timber displayed the colors, but I totally understand if it’s not a priority.
Let me know if we should move this discussion to a separate channel/thread. Thanks!
@jesse Yep for sure will do all this. Just added the endpoint for your / Heroku’s log drains Friday because @slouchpie wanted it. Been focused on Cloudflare & Vercel users. Had an Elixir Logger backend for a while because it’s written in Elixir but there is more interest recently from Elixir users. Anyway, I’ll ping you when this is done!
We use all of them listed here.
@jesse Okay the new Logplex friendly syslog parser is now live. Less than a minute from singing up for Logflare to seeing a Gigalixir log event in Logflare!
The endpoint looks like:
Here’s a run-through:
Everything is getting parsed as you’ll see. And when there is JSON in the message that gets parsed into it’s own object as well. ANSI colorizing is not there yet, but coming soon.
@slouchpie oh, please update your log drain to use
/logs/logplex instead of
/logs/syslog. Logplex is slightly different than the spec so we’d like to use a special endpoint for that and reserve the generic one for the exact spec.
Hats off to you, @chasers
I see the raw ANSI escape sequence scattered throughout the message and raw message fields but the metadata is very very nice! Amazingly fast work and unexpected engagement with users - big love to you and whoever else is working on this awesome tool.
Replying to myself for posterity: don’t use this
syslog URL. instead use