santosh79

santosh79

RemixDB - an Elixir implementation of the Redis protocol

Hi All,
I have just released a project that I had been hacking on for quite a while and would love to get your feedback. It’s basically redis implemented in Elixir. The line level protocol stays the same, this way people currently using Redis who’d (maybe) wish to switch to Remixdb, needn’t have to install new drivers/packages/what have you.

It’s still super experimental and I have some crazy ideas for doing real time sharding, moving processes across nodes in a cluster etc., but it’s no where near being reality, yet.

The repo is at – GitHub - santosh79/remixdb: Key/Value Store focussed on Concurrency and Distribution Primitives · GitHub.

Best,
Santosh

First Post! Switch mode

findlaech

findlaech

That’s awesome, this project has all my attention :smiley:

Most Liked

benwilson512

benwilson512

Author of Craft GraphQL APIs in Elixir with Absinthe

Your readme promises a lot, but your code doesn’t back it up. You need to be clearer in the readme that this is wildly alpha software.

Couple of points that raise some alarms:

  • https://github.com/santosh79/remixdb/blob/master/lib/remixdb.ex doesn’t create an OTP application
  • No supervisors. Not only is this bad practice, it goes against all general claims to fault tolerance in the Readme, as well as against explicit claims to use supervisors.
  • No distributed functionality, despite claiming so in the readme.
  • Your latest commit says “Using HashDicts for better hash performance” and the commit shows you’ve replaced a bunch of Maps with HashDicts. Not only is this false, Maps perform better in literally every case, but HashDicts are deprecated.
  • you’re creating an escript, but a release would be more appropriate for a long running service.
  • extensive use of spawn_link instead of using OTP behaviours.
  • unlinked processes.
  • You aren’t following elixir conventions for file organization

I’m not trying to discourage you, I think there’s a lot of value in thinking through how a key value store in Elixir might work. However, right now your readme is 100% aspirational, but written as if it reflects the current capabilities of your code. You do yourself and your users a disservice by not being clear on this point.

sasajuric

sasajuric

Author of Elixir In Action

Having a Redis protocol compatible k-v, but with an Elixir implementation certainly sounds interesting!

However, I’d advise against hand building the distributed k-v part yourself, as it can become hard to get proper CP or AP guarantees in a clustered environment. Instead, I’d suggest looking for third party libraries which solve those problems. The first ones which come to mind are riak_core for eventual consistency, and riak_ensemble for strong consistency. I also wonder whether recent work on Phoenix presence (i.e. CRDT) could be used for a general purpose distributed k-v, and what are the trade-offs. I’ll ping @chrismccord to hear his thoughts on this :slight_smile:

However, if you just want to practice and/or have fun, then it will probably be simpler to build from scratch, as long as you’re not optimistic about proper behaviour on netsplits. Getting that part right requires carefully applying some scientific papers and a good amount of testing. Otherwise various subtle errors might occur, and the storage might have unpredictable behaviour on netsplits.

I agree with other comments, especially those made by @benwilson512. An app should be OTP compliant, because strange bugs may occur, and it might not play nicely with the client’s project. If you’re still practicing, it’s fine to violate those principles and taking baby steps in whichever order suits your flow. However, the project Readme should clearly state that, because otherwise people may get false impression about the maturity of the library. Here’s an example of how we did it for our Phoenix socket client.

Best of luck!

Qqwy

Qqwy

TypeCheck Core Team

Yes. This used to be the case, but in recent versions of Erlang (18+) this has been changed. Maps are now more performant than HashDicts. Also see http://www.theerlangelist.com/article/eia_elixir_12

Last Post!

santosh79

santosh79

Hi Steven,
I’ve been working on a branch and haven’t pushed up stuff yet. Will be
doing it shortly. Still under active maintenance.

Best,
Santosh

Where Next?

Trending in Announcing Top

bluzky
You may know https://ui.shadcn.com/, a UI component library for React. I really love it’s design style and components. I’ve built some co...
385 14863 120
New
JesseHerrick
Hey, I’m Jesse and I’m the main contributor behind Dexter, a full-featured, lightning-fast Elixir LSP optimized for large codebases. It s...
New
shahryarjb
The Chelekom project is a library of Phoenix and LiveView components generated via Mix tasks to fit developer needs seamlessly. One of i...
New
jimsynz
Beam Bots (or just BB for short) is a framework for building fault-tolerant robotics applications in Elixir using familiar OTP patterns. ...
New
Damirados
Hello everyone. After busy few months I am happy to announce v0.1.0 of Emerge & Solve. They are GUI (Emerge) and State management (S...
New
ausimian
Emily is an Elixir library that runs Nx computations on Apple’s MLX. Install it as the default Nx backend and Nx, defn, Axon, Nx.Serving,...
New
zachdaniel
Introducing AshStorage! Attachment and file management that slots directly into your resources :smiling_face_with_sunglasses: I had hope...
New

Other Trending Topics Top

type1fool
I just stumbled on a newly redesigned elixir-lang.org. :tada: It looks like @Software_Mansion did the work, and I think it is generally a...
New
akoutmos
@hugobarauna and I (Alex Koutmos) have been hard at work on writing a book on Nerves that takes you from simply blinking LEDs to building...
New
juhalehtonen
There has been a thread to discuss the Stack Overflow Developer Survey on this forum every year since 2018, so here’s yet another one for...
New
bjorng
We want to introduce a new native datatype to Erlang: native records. Although replacing all tuple records with native records is not our...
New
spammy
I’m looking to build a personal workflow to quickly deploy web applications written in elixir/phoenix, for local consumption (ie not on t...
New
yureehuh
Introduction Founded in 2017 by landscape ecologist and fire mitigation expert Harry Statter, Frontline developed the first fully integra...
New

We're in Beta

About us Mission Statement