christhekeele

christhekeele

Mnemonix: a generic key-value store adapter library [v0.8.1]

Mnemonix

After a couple years of playing with the concept and learning my way around ever-evolving idiomatic Elixir and venerable OTP patterns, I’ve finally sat down and implemented an Elixir library with an architecture I’m pleased with.

Rubyists might be familiar with Moneta; this is my Elixir approach to that. Mnemonix is a standardized Map-compatible API for interacting with key-value stores in an implementation-agnostic way.

The intent is to define an abstract API a key-value store should offer, and iron out discrepancies between implementations and Elixir wrappers such that they all comply and behave predictably behind it.

If you’re developing an application, it lets you trivially experiment with different backends to find the right implementation for your use-case, and unifies access to different backends chosen for different reasons behind a common, familiar API.

If you’re building a library, it lets you defer the implementation decision of what key-value store to support to your end-users.


Current State

As of v0.7.1, I’m beginning to move towards optimization and better error reporting. The public API has now stabilized and only new capabilities will change public function signatures.

Currently, it offers parity with the Map module, and builds upon that with extra functions for common key-value store needs: incrementing and decrementing integer values, and setting explicit keys to expire after so many seconds.

It supports several backends:

  • Map
  • ETS
  • DETS
  • Mnesia
  • Redis
  • Memcache

I intend to improve the existing backends, add a few more, and add a few more features to all stores before v1.0.0.

Check out the documentation to learn more, let me know of any issues you run into!

Most Liked

jxxcarlson

jxxcarlson

I used Mnemonix tonight to solve a problem I had with the app I am working on. Mnemonix is a really useful and elegant tool. I put this code

{:ok, store} = Mnemonix.Store.start_link({Mnemonix.Map.Store, initial: %{active_notes: []}}, name: Cache)

in the APP_NAME.exs file and within two hours the job was done. Three cheers for Mnemonix!

jxxcarlson

jxxcarlson

Hi Chris, some info about my use case, of which there are two, both in Phoenix – one present and one future.

  1. Lookup.

This is an app for storing short snippets of information – from things like the speed of light and Planck’s constant to links to various things I find on the web: political articles, github repositories, articles about code, you name it. Just two fields: title and description. URLs in the description field are rendered as links, e.g, http:foo.bar.io/yada/yada?baz=123 is rendered with link text “foo.ba.io”.

I use Mnemonix at the moment for just one thing: to record a list of IDs of records in the database result from the most recent relevant operation – search, create, edit, etc. In the case of deleting a record, the list is the empty one. This is needed to give the user a pleasant and efficient experience.

I plan to add users and authentication and then put this up on the web for all to enjoy. It will also help me, since I will then have access to my data anywhere there is a connection. I plan to use Mnemonix to hold the user’s JWT authentication token, the id list, and maybe more, e.g, preferences.

The code is at GitHub - jxxcarlson/lookup_server: A Phoenix server for Lookup · GitHub

  1. Rewrite the backend for Manuscripta.io:

http://www.manuscripta.io/

Manuscripta is an app for creating, editing, and distributing lecture notes, although it can be used for other things. For writing techical docmentation (code, math, physics), it uses Asciidoctor-LaTeX (GitHub - asciidoctor-contrib/asciidoctor-latex: 📐 Add LaTeX features to AsciiDoc & convert AsciiDoc to LaTeX · GitHub), of which I am the principal developer.

Here is an example --my course notes – http://www.manuscripta.io/documents/jc.qft?toc

I am a refugee from the Rails world. I used Rails for the first version of Manuscripta, but (a) it was too slow, (b) the code got completely out of control. Partly my fault. It was my first real software development experience. Subsequently I split the app into a REST back end written with Hanami, a Ruby web framework which I really like (http://hanami.org). The current front end is written in Angular1 and I have an Angular2 version in the works. But I love elixir/phoenix and have found the functional programming style to my liking (I used to program in Scheme some time ago). Hence the plan to redo the backend in Phoenix. (1) Above was my starter project to learn Elixir and Phoenix. (I also wrote a command-line version of lookup – GitHub - jxxcarlson/Lookup: Command line program (Elixir) to create and search for records of the form [title, note] · GitHub)

One challenge (for me) of the backend project is that Phoenix will have to communicate with other processes: ruby for rendering Asciidoc-LaTeX documents into HTML and/or LaTeX, and for converting (when necessary) LaTeX docs to PDF.

I plan to rewrite the front end in Elm, but if you have thoughts on front end technologies, I am interested.

This is probably way too much info, but there you go!

PS – I’ll comment on docs, etc. tomorrow. Late here in Ohio!

christhekeele

christhekeele

I finally found some time to pick this back up again, after a busy few months. v0.9.0 is now released! It’s another small public API change:

  • The start_link functions on Mnemonix.Store have migrated to the main Menmonix module.
  • The Mnemonix.Store.Supervisor module has been renamed to Mnemonix.Supervisor.

These changes are in service to finally fully hiding the Mnemonix.Store namespace as a private API, making usage of v0.9.0 and above much more resilient to API changes in the future.

Furthermore, the start_link functions in Mnemonix are now furnished by the new Mnemonix.Features.Supervision module, which is also used in the Mnemonix.Builder module. This means if you’re creating your own store modules, you no longer need to implement start_link yourself.

Finally, all functions provided by the Mnemonix.Builder are overridable, meaning if you are fluent in the Mnemonix feature APIs you can redefine them for your custom store modules yourself. This also means if you already implemented start_link yourself you do not need to make any changes.

I know it’s more bookkeeping than exciting features. However:

  • It unifies almost all Mnemonix functions and grants them to custom stores. That work should be complete with a final addition of setting a default adapter in the builder, allowing the new functions to move into their own feature and make their way into custom stores. This will be a backwards-compatible change.

  • It finalizes the module namespaces for the project’s initial v1.0.0 release, allowing me to make some of the really fun optimization changes with confidence behind Mnemonix.Store.

Also up for the next release is adding warning infrastructure to the private Store API, allowing store implementations to choose to support a feature, even if it is unwise, but emit warnings in the caller when it does so (rather than just having the option to continue or raise). This is in service to allowing stores that you probably shouldn’t enumerate over to be enumerable none-the-less.

Where Next?

Popular in Announcing Top

deadtrickster
I’ve just released stable versions of my Prometheus Elixir libs: Elixir client [docs]; Ecto collector [docs]; Plugs instrumenter/Export...
New
danschultzer
In short Plug n’ play OAuth 2.0 provider library. Just set up a resource owner schema with Ecto (your user schema), install the dependen...
New
sorentwo
Hello! tl;dr Announcing Oban, an Ecto based job processing library with a focus on reliability and historical observability. After spen...
985 43487 311
New
dbern
I’m excited to announce that TaxJar has developed and open-sourced DateTimeParser. We developed it because we found a need to parse user ...
New
zorbash
I created Kitto a framework for dashboards inspired by Dashing. The distributed characteristics of Elixir and the low memory footprint...
New
Qqwy
Hello everyone, I wrote a small library today called MapDiff. It returns a map listing the (smallest amount of) changes to get from map...
New
Hal9000
Here is my first stab at this. README pasted below. https://github.com/Hal9000/elixir_random Comments and critiques are welcome. Thank...
New
anshuman23
Hello all, I have been working on my proposed project called Tensorflex as part of Google Summer of Code 2018.. Tensorflex can be used f...
New
handnot2
Samly can be used to enable SAML 2.0 Single Sign On in a Plug/Phoenix application. This library uses Erlang esaml to provide plug enabl...
New
trisolaran
Hi! :waving_hand: I would like to present LiveSelect, a little library that I wrote to easily add a dynamic selection input to your LV f...
198 10954 107
New

Other popular topics Top

malloryerik
Hi, this is for people who, like me, have had some friction using .html.heex templates in VSCode. The solution seems to be, in a hyphena...
New
lastday4you
I wanted to check elixir version in phoenix because i found that my elixir is 1.5 but when i use Enum.chunk_by it said the function is un...
New
TunkShif
This post is an instruction guide to help you setup your Neovim for Elixir development from scratch. It includes general information on h...
274 41989 114
New
albydarned
Hello all! I am typing this post from my new MacBook Pro with the M1 chip. I’m loving it so far, and will probably use it as my daily dr...
New
gshaw
What is the idiomatic way of matching for not nil in Elixir? E.g., First way: defp halt_if_not_signed_in(conn, signed_in_account) when...
New
electic
Hi, I am new to Elixir. I am trying to use the DateTime component to insert a date into MySQL however the there seems to be no way to fo...
New
komlanvi
Hi everyone, I was playing with phoenix liveView but I run into an issue. I have a form and want to validate each input text when the te...
New
hariharasudhan94
Lets say I have map like this fetching from my database %{"_id" => #BSON.ObjectId<58eb1a7a9ad169198c3dXXXX>, "email" => ...
New
vegabook
I’m brand new to Phoenix and I have stripped one of the demo applications to the bone. I just want to get an svg up on the screen. Here i...
New
sergio
Kind of like when jquery came out, it was super necessary. Existing drag and drop libraries have a bunch of baggage to support old browse...
New

Latest on Elixir Forum

Elixir Forum

We're in Beta

About us Mission Statement