Przemek

Przemek

EventLog - append log for events

Hi,

I’ve just published my tiny library:

https://github.com/pbudzik/eventlog

It’s an indexed log for events inspired by some Kafka concepts. It models a “stream” as a log and its offset index. It can append, read sequentially from a given offset. Seems to work just fine, however I’d like to improve write throughput. Simple benchmark shows ~8K events at 4 MB/s. Without indexing it’s a few times better.

Profiling kind of shows it’s send/call and I am aware that message passing is costly.

If any of you could see and play with my crap maybe we could improve it together :slight_smile:

regards,
PB

Most Liked

jdumont

jdumont

Hey!

I’ve just spent a bit of time looking through your library and getting to grips with how it works internally. It looks like a very similar solution to what I’ve been building myself, albeit nicely packaged up as a standalone lib. Mine’s currently all tangled up with the work-in-progress memory image app that I’ve been working on (thread about it here).

Where mine is a bit different is that I keep the index in memory with the ability to rebuild it directly from the log. It’s based on the KV store built in this series of articles, although I’ve made some changes to suit streaming events from a given offset rather than being a KV.

Not sure how that impacts performance as I’ve not properly benched it, but when I next get a chance I’ll swap my log out for yours, and perhaps package mine up nicely and get some tests/benchmarks around it.

Thanks for publishing this. It’s great to take a look at another way of solving the problem. I’ll be sure to return the favour once mines a little more presentable! :+1:t2:

Where Next?

Popular in Discussions Top

AstonJ
If a newbie asked you about Phoenix Contexts, how would you explain the basics to them? Feel free to be as concise or in-depth as you li...
New
acrolink
How does the two languages compare when it comes to server side application development? Any experiences or ideas? Thank you.
New
jer
I’ve been using umbrellas for a while, and generally started off (on greenfield projects at least) by isolating subapps based on clearly ...
New
pdgonzalez872
If this has been asked here before, please point me to where it was asked as I didn’t find it when I searched the forum. Maybe a mailing ...
New
Nvim
Anybody knows a comprehensive comparison of Django and Phoenix, thanks for the help. Where are they similar? Where do they differ the m...
New
saif
Hello everyone, Long time lurker first time poster here. I’ve recently begun working on Elixir full-time again! :raised_hands: It’s been...
New
crispinb
On reading dhh’s latest The One Person Framework it strikes me that Phoenix with LiveView is already pretty much this. However, never hav...
New

Other popular topics Top

electic
Hi, I am new to Elixir. I am trying to use the DateTime component to insert a date into MySQL however the there seems to be no way to fo...
New
joeerl
Hello again - after a longish gap I’ve decided I really must dig into Elixir and see what’s been happening here - so I have a few questio...
New
sorentwo
Hello! tl;dr Announcing Oban, an Ecto based job processing library with a focus on reliability and historical observability. After spen...
985 44778 311
New
siddhant3030
Hi, I have to write a raw query for one of my project. But till now I have used ecto queries and don’t have much experience writing raw ...
New
saif
Hello everyone, Long time lurker first time poster here. I’ve recently begun working on Elixir full-time again! :raised_hands: It’s been...
New
AngeloChecked
What learn first? Rust or Elixir Hi Elixir community! I’m here because i want learn a new language. I’m a junior developer and mainly i ...
New

We're in Beta

About us Mission Statement