michalmuskala

michalmuskala

PersistentEts - lightweight persistence over ets

Another small library today.

PersistentEts

Hex: persistent_ets | Hex
GitHub: GitHub - michalmuskala/persistent_ets · GitHub

Ets table backed by a persistence file.

The table is persisted using the :ets.file2tab/2 and :ets.tab2file/3 functions.

Table is to be created with PersistentEts.new/3 in place of :ets.new/2. After that all functions from :ets can be used like with any other table, except :ets.give_away/3 and :ets.delete/1 - replacement functions are provided in this module. The :ets.setopts/2 function to change the heir is not supported - the heir setting is leveraged by the persistence mechanism.

Like with regular ets table, the table is destroyed once the owning process (the one that called PersistentEts.new/3) dies, but the table data is persisted so it will be re-read when table is opened again.

Example

pid = spawn(fn -> 
  :foo = PersistentEts.new(:foo, "table.tab", [:named_table])
  :ets.insert(:foo, [a: 1])
end)
Process.exit(pid, :diediedie)
PersistentEts.new(:foo, "table.tab", [:named_table])
[a: 1] = :ets.tab2list(:foo)

Most Liked

michalmuskala

michalmuskala

With Dets every operation (read or write) hits the disk. For many application such a performance penalty (compared to ets) is not acceptable. Furthermore Dets tables are limited to 2GB. Dets doesn’t support the ordered_set table type either.

With PersistentEts, the table remains in memory, so all read and write operations have the same performance they would have with pure ets. Only periodically the table state is saved to a file. There’s also no file limit, besides the memory and disk limitations. Since it’s a regular Ets table, all types are fully supported.

hubertlepicki

hubertlepicki

Could you highlight for us how it differs from DETS and why would someone choose one against another?

hubertlepicki

hubertlepicki

Copy paste that to readme now.

Last Post!

lessless

lessless

Thank you for insights!

I tried to look into the sources of tab2file but didn’t have enough patience to dig deep down into what they’re using there for file manipulation :smile:
I just saw that they’re using message passing and that was enough for me to understand that there will definitely be overhead.

Where Next?

Popular in Announcing Top

Hal9000
Here is my first stab at this. README pasted below. https://github.com/Hal9000/elixir_random Comments and critiques are welcome. Thank...
New
KronicDeth
Elixir plugin for JetBrain’s IntelliJ Platform (including Rubymine) This is a plugin that adds support for Elixir to JetBrains IntelliJ...
289 36820 110
New
wmnnd
Hi there, for my project DBLSQD, I needed a file storage solution that is a bit more flexible than Arc. Because I thought others might f...
New
kelvinst
Hey everyone! Well, we made this lib a while ago and now we decided to finally go out and public with it! It’s a tool for creating and m...
New
mspanc
I am pleased to announce an initial release of the Membrane Framework - an Elixir-based framework with special focus on processing multim...
New
tfwright
After working on it for a couple of months and using it in production for most of that time, today I’ve released LiveAdmin, a LiveView ba...
New
martinthenth
Hello everybody :wave: Recently, some of my colleagues talked about database ids and uuids and their problems, and I remembered the pain...
New

Other popular topics Top

hariharasudhan94
Lets say I have map like this fetching from my database %{"_id" => #BSON.ObjectId<58eb1a7a9ad169198c3dXXXX>, "email" => ...
New
lanycrost
Hi everyone! I need implement if…else if…else condition from my elixir code, and anymore of this control flow structures not work proper...
New
ashish173
I am using Ecto timestamps with postgres, I can see the timestamps() use the :naive_dateime but for my use case I wanted to store the ti...
New
sergio_101
I am VERY much an elixir newbie. I have taken one elixir course and one phoenix course on Udemy. During that course, I saw the instructor...
New
SoCreat
i’m a new one to elixir which editor can i use vs code? or atom? Thanks! :smiley:
New
Harrisonl
We have an ECS cluster with 4 services, where each task joins a single cluster, via discovery ECS discovery service. Currently when I de...
New

We're in Beta

About us Mission Statement