alvises

alvises

New Series: Build a Simple Persistent Key-Value Store in Elixir, using Logs

Just published the first part of series of articles aimed to explain the architecture behind a kv-store engine written in Elixir and implemented with logs. During the series I describe (and implement in Elixir just for the purpose of inspection) the concepts of logs and data immutability, sequential writes, index, segments, compaction, memtable, sstable etc..

In this first part:

  • What is a log?
  • Making a KV persistent using a log and an index. Using an example we will use along the series, crypto market prices and trades, we are going to see how to store the values in a log using using an index.
  • LogKV in Elixir. A initial super simple implementation in elixir of a Writer, an Index and a Reader.

Please let me know what you think, available here on the forum as @alvises or on Twitter as @alvisesus

Most Liked

alvises

alvises

Second part of the kv-store engine series. In this part I show how to do a step further, making both keys and values persistent. Taking inspiration from the Bitcask design, I describe how to enhance our implementation and to recover the Index in the case of a failure.

Where Next?

Popular in Blog Posts Top

alexgaribay
I saw this thread so I decided to write about how we do it :smile:
New
jordiee
https://medium.com/@jpiepkow/distributed-state-is-hard-5a0d384c2f3c
New
AstonJ
Elixir Blog Posts How to use this section You can post links to your blog posts either in one of the Official Blog Posts threads, or, vi...
New
rlopzc
Learn how to leverage Erlang + Elixir’s GenServer to build a robust FTP Client. https://rlopzc.com/posts/elixir-ftp-client-genserver-par...
New
rms.mrcs
https://medium.com/the-realreal-product-tech-blog/monitoring-hackney-connection-pools-with-telemetry-3aaeafa8eeb8 Hi, everyone! In this ...
New
brainlid
OTP 26 was released and the Elixir 1.14.4 builds have been updated! Erlang OTP 26 changed how map keys are sorted, or not sorted actually...
New
wmnnd
Hey there, I’ve started a little blog series about building and deploying Elixir applications. Now I would like to share with you the fi...
New
JEG2
I’m closing out, for now, my series on questions at the heart of development with an analysis of when we need more abstraction. For exam...
New
mssantosdev
This post is a guide on how Norba and I are building LiveMatch, a real-time app for soccer to follow multiple games in one place. https:...
New
brainlid
Dialyzer is a tool that you’ve probably heard about in the Elixir community. You may have even used it. However, adding Dialyzer to an ex...
New

Other popular topics Top

sen
Hi All, I set a environment variables in dev.exs , like below code. when i start server, how can i set the ${enable} value? thanks. d...
New
senggen
Erlang/OTP 25 [erts-13.2.2] [source] [64-bit] [smp:8:8] [ds:8:8:10] [async-threads:1] 15:22:35.803 [error] gen_event {lager_file_backend...
New
skosch
To my knowledge, put_in, Map.update etc. all have the one limitation of not automatically creating intermediate keys when needed (for exa...
New
lessless
I believe there are people here who are dealing with CSV files import on the daily basis, and since Excel is a really popular tool there ...
New
greenz1
I have a phoenix application from which a user can download multiple(5-6) files of size 1MB. I couldn’t find anything related to sending ...
New
Patoshizzle
After calling mix ecto.create I get this error: 17:00:32.162 [error] GenServer #PID<0.412.0> terminating ** (Postgrex.Error) FATAL...
New
gausby
I asked this very same question on twitter and got some interesting feedback, but I thought it would be a good question to ask here as we...
1207 39297 209
New
RisingFromAshes
I’ve read in another post that it may be possible with a router helper - but I couldn’t find an appropriate one, and tbh, I’m still just ...
New
KronicDeth
Elixir plugin for JetBrain’s IntelliJ Platform (including Rubymine) This is a plugin that adds support for Elixir to JetBrains IntelliJ...
289 36128 110
New
boundedvariable
I am going through the kafka architecture. All the features what the kafka is providing are already in Erlang. I would like hear your opi...
New

We're in Beta

About us Mission Statement