chasers

chasers

Anyone interested in another logging thing?

I’ve been building out Logflare with Elixir, which has been great. Initially for Cloudflare customers via the Cloudflare app, but doing this made me want a bit more out of the Elixir logging ecosystem.

Announcing the Logflare Logger backend!

You can easily pump your structured (or not) Elixir logs to Logflare, which then automatically get inserted into a BigQuery table you can query and own directly.

We’ve been using it on production ourselves for a few weeks now. Here is a dashboard I’m using (we need to upgrade our box soon actually).

The whole idea really is that you should log structured data, and you should be able to log liberally into an inexpensive datastore that you can query over the long term. So essentially Logflare is just a fancy queue into BigQuery. Ultimately we should be able to do this a lot cheaper than your standard ELK based log management system because we’re not going to be marketing up the storage at all and BigQuery is way cheaper than Elastic. You Bring Your Own Backend™, and just pay Logflare to get the data there.

I don’t really want to get into the whole logs vs metrics debate because everything breaks at some scale and it’s always good to use the right tool for the job, but I think 95% of apps could just log structured events to a database instead of using statsd-like things. This gets you a ton of benefits over a timeseries db. Honeycomb.io, for example, is also pushing event-based metrics over a statd-like system.

I also am not saying telemetry is bad, because it’s awesome and I think Elixir needs standardized monitoring hooks. We actually hope to have a telemetry backend soon. It’s more the underlying storage I have an issue with, which is also why we ultimately want you to be able to use other databases than BigQuery (eventually).

Anyways, would love some feedback on this whole thing. Our Cloudflare users seem to really like it, and I’m hoping the Elixir community will too.

ps: None of this would even be possible without everything that Elixir/Phoenix is. I’ve been in the software business for over 10 years and I haven’t been this excited about something in a very long time.

pps: We should have LiveView based search done next week:

Showing Posts 1 to 10

chasers

chasers OP

Logflare Logger Backend is now compatible with new context keys for Elixir 1.10.

I should probably post about this a bit more but in terms of the product Logflare itself, we’re now basically feature matched with something like Papertrail (in terms of streaming, searching, alerting and user access). Working on billing now, and are planning to go v1 once billing is launched. Oh, and we have a Zeit integration also if anyone has any (non-Elixir) stuff on Zeit.

Would love some feedback from Elixir folks!

chasers

chasers OP

And now you can send your browser security reports to Logflare!

Set your Report-To url to https://api.logflare.app/logs/browser/reports?api_key=KEY&source_id=SOURCE and we’ll take in everything the reporting API sends over. So ideally you can do something like this for CSP, NEL, crash, intervention etc reports:

https://p195.p4.n0.cdn.getcloudapp.com/items/5zuyB9wk/Screen+Shot+2020-03-06+at+10.22.20+AM.png?v=bf904058cbb874ae61b8b0bde5edf00e

hauleth

hauleth

Noice, thanks about letting me know about Report-To, unfortunately this seems to not be available in any non-Chrome-based browsers :frowning:

chasers

chasers OP

Big update to our search UI … we now have a Date/Time picker, and the whole form is an actual query builder.

One really cool part of this is that if you have the Logflare UI set to local time it automatically converts your query to UTC.

It’s super easy to do timestamp ranges with t:2020-04-10T01:{50..55}:00

And we support microsecond time ranges like t:2020-04-08T07:40:02.{000001..999999}

https://p195.p4.n0.cdn.getcloudapp.com/items/4guy0P1w/Screen%20Recording%202020-04-10%20at%2012.13%20PM.gif?v=0fa3309471a0006b632fc2c61e723821

Now to finish billing…

Crowdhailer

Crowdhailer

Creator of Raxx

This looks very cool, I wonder if I can mangle it to try it out in a Gleam project.

I need to try out something new after being on timber.io for a while.

chasers

chasers OP

Thanks! I have not played with Gleam yet. Would love to know how it goes!

Jcambass

Jcambass

How does the logger backend handle network problems? Does it just noop or fallback to stdout? I‘ve experienced some app crashes today when logflare had some issues. Quite possible that I also did some suboptimal things which lead to this so for the lack of debug information I didn‘t want to open a GH issue yet.

chasers

chasers OP

Please upgrade to the latest version we released last week. It fixes this issue. Previously issues POSTing to Logflare could have put Logger into sync mode which would definitely affect things negatively. Sorry for the trouble!!

Jcambass

Jcambass

Great to hear that this is already addressed! No worries :grin:

chasers

chasers OP

logflare_logger_backend has been updated to v0.8.1.

We’ve updated the deps and validated Elixir 1.12 and OTP 24 compatibility. Logflare prod is upgraded and sending logs to our staging env successfully.

Ideas for improvements very much welcomed :slight_smile:

Where Next? Top

Trending in Discussions Top

AstonJ
The obligatory hello world thread! Who are you and where are you from? :stuck_out_tongue:
4663 61218 595
New
cblavier
Hey there, It’s been more than a year since we started using LiveView as our main UI library and building a whole library of UI componen...
515 12931 101
New
sntran
In my third year of college, around 2010, I found Erlang. A few years later, I found Elixir. I liked the ideas at once: many small proces...
New
gtcode
I’ve been tinkering with Lynx (by José) via hold_ledger. I used AI to write the code, a separate specification, the tests, and the Lean p...
New
ManuelGarcia
Hi everyone! I’m the developer of PaperForge, and I’ve been working on something alongside it that I’d like your opinion on. I’m conside...
New
AstonJ
A remark @Garrison made… ..has already inspired a thread for the technical side of reclaiming our internet, but there is of course anot...
New
bartblast
This is where the projects in the Hologram Showcase get posted: apps built with Hologram and libraries made for it. New ones go up usuall...
New

Other Trending Topics Top

GenericJam
Edit: 2026 May 15 - This post is archived. Mob is alive!! Main docs: mob v0.7.11 — Documentation A bit of explanation for the slightly c...
261 13183 112
New
mudasobwa
I fully migrated to my own harness from Anthropic/Gemini and I think it’s time to share it. Welcome DSH, the DeepSeek Harness, fully writ...
New
OndrejValenta
(I just needed to vent somewhere and LinkedIn is full of hope, or hype, I’m not sure which exactly) AI dream has many faces, but general...
New
kip
Localize is the next generation localisation library for Elixir. Think of it as ex_cldr version 3.0. The first version will be released ...
New
jimsynz
Beam Bots (or just BB for short) is a framework for building fault-tolerant robotics applications in Elixir using familiar OTP patterns. ...
New
mcass19
ExRatatui lets you cook up rich terminal UIs in Elixir, powered by Rust’s ratatui via Rustler NIFs. Build interactive terminal applicatio...
New

We're in Beta

About us Mission Statement

Options

Thread Display Mode




Thread Preview

Skip Thread Previews