lcabrini

What would be the “proper” way to implement a simple persistent key/value store?

What would be the “proper” way to implement a simple persistent key/value store? Settings that are read far more than they are written. In Go I used a single postgres table and on application start-up I read it into to a mutex-backed struct. The write method would persist the setting in the database.

I wouldn’t necessarily need to use Postgres as a backing store, but that way the settings get backed up with the rest of the data. That being said, I’m not against using a non-postgres backing store, if that makes sense.

I’ve looked a bit into ETS, Mnesia, I also read somewhere that it would be a good idea to build a GenServer around a key/value store. I feel a bit overwhelmed by the choices and would like a pointer in the right direction. Thank you.

40 comments

/phoenix

1 1084 40

2026-01-04 06:03:03 UTC

Most Liked

garrison

If you’re using Postgres, use Postgres. Otherwise use SQLite. Create a table with binary keys/values and then use ETS as a write-through cache.

Always write to the DB first and then the cache before returning from the write function. Always read from the cache.

def put(key, value) do
  Repo.insert! %Row{key: key, value: value}
  :ets.insert(@table, {key, value})
end

def get(key) do
  case :ets.lookup(@table, key) do
    [{^key, value}] -> value
    [] -> nil
  end
end

Load the rows from the DB into the cache at startup. If you want to use arbitrary terms just encode them with term_to_binary().

Edit: the put() function above is only correct for a single writer, meaning it cannot be used from multiple processes without synchronization. See @Asd 's more thorough answer below for an example that uses a GenServer as a single writer to serialize writes.

Post #7

Asd

Yeah, you’re right, if its a single writer, then your code is correct.

Why is it always the same people in KV DB topics . Happy new year btw!

Post #12

Asd

I’d do this like this if you want it backed by postgres table

defmodule Table do
  @moduledoc "ets table backed by postgres table"

  use GenServer

  def start_link(opts) do
    GenServer.start_link(__MODULE__, opts, name: __MODULE__)
  end

  def init(_opts) do
    table = :ets.new(:table_name, [:protected, :set, :named_table])
    entries = for %{key: k, value: v} <- Repo.all(Table), do: {k, v}
    :ets.insert(table, entries)
    {:ok, %{table: table}}
  end

  def handle_call({:write, key, value}, _from, %{table: table}) do
    Repo.insert!(%Table{key: key, value: value})
    :ets.insert(table, {key, value})
  end

  def read(key) do
    case :ets.lookup(:table_name, key) do
      [{_, value}] -> {:ok, value}
      _ -> :error
    end
  end

  def write(key, value) do
    GenServer.call(Table, {:write, key, value})
  end
end

But if you’re okay with just a file on disk, I’d consider using a dets

However, I am going to release much more performant persistent LVM KV db in the upcoming months, so I will reply here again once it’s ready

Post #9