michalmuskala

michalmuskala

PersistentEts - lightweight persistence over ets

Another small library today.

PersistentEts

Hex: persistent_ets | Hex
GitHub: GitHub - michalmuskala/persistent_ets · GitHub

Ets table backed by a persistence file.

The table is persisted using the :ets.file2tab/2 and :ets.tab2file/3 functions.

Table is to be created with PersistentEts.new/3 in place of :ets.new/2. After that all functions from :ets can be used like with any other table, except :ets.give_away/3 and :ets.delete/1 - replacement functions are provided in this module. The :ets.setopts/2 function to change the heir is not supported - the heir setting is leveraged by the persistence mechanism.

Like with regular ets table, the table is destroyed once the owning process (the one that called PersistentEts.new/3) dies, but the table data is persisted so it will be re-read when table is opened again.

Example

pid = spawn(fn -> 
  :foo = PersistentEts.new(:foo, "table.tab", [:named_table])
  :ets.insert(:foo, [a: 1])
end)
Process.exit(pid, :diediedie)
PersistentEts.new(:foo, "table.tab", [:named_table])
[a: 1] = :ets.tab2list(:foo)

Most Liked

michalmuskala

michalmuskala

With Dets every operation (read or write) hits the disk. For many application such a performance penalty (compared to ets) is not acceptable. Furthermore Dets tables are limited to 2GB. Dets doesn’t support the ordered_set table type either.

With PersistentEts, the table remains in memory, so all read and write operations have the same performance they would have with pure ets. Only periodically the table state is saved to a file. There’s also no file limit, besides the memory and disk limitations. Since it’s a regular Ets table, all types are fully supported.

hubertlepicki

hubertlepicki

Could you highlight for us how it differs from DETS and why would someone choose one against another?

hubertlepicki

hubertlepicki

Copy paste that to readme now.

Last Post!

lessless

lessless

Thank you for insights!

I tried to look into the sources of tab2file but didn’t have enough patience to dig deep down into what they’re using there for file manipulation :smile:
I just saw that they’re using message passing and that was enough for me to understand that there will definitely be overhead.

Where Next?

Popular in Announcing Top

Qqwy
Hello everyone, I wrote a small library today called MapDiff. It returns a map listing the (smallest amount of) changes to get from map...
New
wmnnd
Hi there, for my project DBLSQD, I needed a file storage solution that is a bit more flexible than Arc. Because I thought others might f...
New
mspanc
I am pleased to announce an initial release of the Membrane Framework - an Elixir-based framework with special focus on processing multim...
New
tmbb
I’ve been working on two packages (not on hex.pm yet) to build admin interfaces for phoenix apps: bureaucrat - which contains a bunch ...
New
aesmail
Hello guys, I have finally made it. I created an admin interface for a framework. It’s been on my todo list for years and with the curre...
New
restlessronin
The repo is at GitHub - cyberchitta/openai_ex: Community maintained Elixir library for OpenAI API · GitHub. Docs are at OpenaiEx User Gu...
152 10796 134
New
brainlid
LangChain is short for Language Chain. An LLM, or Large Language Model, is the “Language” part. This library makes it easier for Elixir a...
New

Other popular topics Top

Brian
What is the proper way to load a module from a file in to IEX? In the python world, doing something like this pretty standard: from ....
New
stefanchrobot
What’s the safe way to decode a JSON string into a struct? I want to avoid calling String.to_atom. Jason.decode can give me a map with st...
New
greenz1
I have a phoenix application from which a user can download multiple(5-6) files of size 1MB. I couldn’t find anything related to sending ...
New
AstonJ
Seen any cool LiveView demos, sample apps or examples? Please post them here! :003:
New
alice
Hey, Just curious what are the main benefits of Elixir compared to Clojure? When is Elixir more useful than Clojure and vice versa? Th...
New
sergio
Kind of like when jquery came out, it was super necessary. Existing drag and drop libraries have a bunch of baggage to support old browse...
New

We're in Beta

About us Mission Statement