RemixDB - an Elixir implementation of the Redis protocol

That’s awesome, this project has all my attention

Very interesting! On your README you say:

Fault Tolerance: Being able to bring back a key-value process when it goes down based on some last known state is trivial.

Can you explain a little bit how you achieve that?

Your readme promises a lot, but your code doesn’t back it up. You need to be clearer in the readme that this is wildly alpha software.

Couple of points that raise some alarms:

• https://github.com/santosh79/remixdb/blob/master/lib/remixdb.ex doesn’t create an OTP application
• No supervisors. Not only is this bad practice, it goes against all general claims to fault tolerance in the Readme, as well as against explicit claims to use supervisors.
• No distributed functionality, despite claiming so in the readme.
• Your latest commit says “Using HashDicts for better hash performance” and the commit shows you’ve replaced a bunch of Maps with HashDicts. Not only is this false, Maps perform better in literally every case, but HashDicts are deprecated.
• you’re creating an escript, but a release would be more appropriate for a long running service.
• extensive use of spawn_link instead of using OTP behaviours.
• unlinked processes.
• You aren’t following elixir conventions for file organization

I’m not trying to discourage you, I think there’s a lot of value in thinking through how a key value store in Elixir might work. However, right now your readme is 100% aspirational, but written as if it reflects the current capabilities of your code. You do yourself and your users a disservice by not being clear on this point.

Thanks for your input. Yes, I’ll be the first to agree the code is not
backing up what’s on the README. But the fact that this is being built in
Elixir means I have somewhat of a shot at making this real.

Ben Wilson - thanks for your suggestions. All of them are valid and I
should be addressing them shortly. To pick just 1 – the reason for the
switch to HashDicts was that “Elixir in Action” said they performed better
than Maps for larger collections. Is this wrong? If so, then yea it makes
sense I didn’t do it.

Again this is super experimental and all these issues such as – Making it
an OTP application, running it as a daemon are all in my list of things to
do. Heck, I need to be able to read a RDB file to be able to load in
existing Redis data!! So yea, I am in no way saying yank out Redis and go
and use this in production right now.

Thanks again for offering your inputs, looking at code.

Best,
Santosh

Yes. This used to be the case, but in recent versions of Erlang (18+) this has been changed. Maps are now more performant than HashDicts. Also see http://www.theerlangelist.com/article/eia_elixir_12

Thanks for sharing that article, I wasn’t aware of the deprecation of
HashDict.

Best,
Santosh

@santosh79,

Elixir in Action is an incredible resource and most of my knowledge of OTP comes from it. I see that you’ve already been pointed to the article that @sasajuric wrote when Elixir 1.2 came out.

Are you trying to create a Redis server? There are many client implementations that you can find redis.io itself, many Erlang and a few Elixir clients.

I appreciate your response here, and I do believe that you had no intention of misleading any one.

My only further comment is to suggest that while some of these features (distribution) are definitely things that you can wait on for now and develop in time, others form the foundation of a proper Elixir package and should be prioritized higher, even over “basic functionality”. Specifically:

• Organizing your code properly. It’s really hard for people to offer advice or contribute if there’s no real structure to the files.
• Using supervisors. This one is crucial. It will also simplify the spaghetti of start_link calls that happen.
• Generate a valid application. mix new with the --sup flag gives you the bare bones, and it’s covered in the getting started guides and virtually every elixir book.

Best of luck to you!

Having a Redis protocol compatible k-v, but with an Elixir implementation certainly sounds interesting!

However, I’d advise against hand building the distributed k-v part yourself, as it can become hard to get proper CP or AP guarantees in a clustered environment. Instead, I’d suggest looking for third party libraries which solve those problems. The first ones which come to mind are riak_core for eventual consistency, and riak_ensemble for strong consistency. I also wonder whether recent work on Phoenix presence (i.e. CRDT) could be used for a general purpose distributed k-v, and what are the trade-offs. I’ll ping @chrismccord to hear his thoughts on this

However, if you just want to practice and/or have fun, then it will probably be simpler to build from scratch, as long as you’re not optimistic about proper behaviour on netsplits. Getting that part right requires carefully applying some scientific papers and a good amount of testing. Otherwise various subtle errors might occur, and the storage might have unpredictable behaviour on netsplits.

I agree with other comments, especially those made by @benwilson512. An app should be OTP compliant, because strange bugs may occur, and it might not play nicely with the client’s project. If you’re still practicing, it’s fine to violate those principles and taking baby steps in whichever order suits your flow. However, the project Readme should clearly state that, because otherwise people may get false impression about the maturity of the library. Here’s an example of how we did it for our Phoenix socket client.

Best of luck!

@StevenXL yes, I’m trying to create redis-server.