Qqwy

Qqwy

TypeCheck Core Team

Fast Mergeable Integer Maps: Maybe interesting to port this to Elixir?

Hello, everyone.

At the University I am regularly working in Haskell to make certain exercises for the Advanced Algorithms and Datastructures cource. A few days ago, I came across Haskell’s Data.IntMap and Data.IntSet containers. These turn out to be ridiculously fast, running most operations (insert, find, delete, merge) in O(min(n, W)) time, where W is 32 for 32-bit machines and 64 for 64-bit machines.

This sounded very intriguing, since it goes against the understanding I had thus far about Hashmaps, which is that they are usually implemented as Red-Black trees (or similar auto-balancing binary trees) or Finger trees (I believe this is what Erlang’s maps are built on?), which take O(log(n)) time for these operations.

Haskell’s IntMap and IntSet, however, are built on so-called Patrizia trees, which store, in their nodes, the binary representation of (a part of the prefix of) the given key. The ‘drawback’ is of course that you cannot have keys larger than 2^W, but for many applications, this is perfectly fine.

The original paper called Fast Mergeable Integer Maps, written by Chris Okasaki and Andy Gill can be found here.
I don’t have the time in the near future to build an implementation of these in Elixir, but I think it would be an intriguing project, since I expect that maps/sets built on these kinds of trees will be faster than Elixir/Erlang’s built-in maps (even though the latter can be optimized by the BEAM), because of the speed that restricting the key type will bring.

I wanted to share this here because I expect that I’m not the only one who finds this interesting, and maybe someone would like to port this to Elixir :slight_smile: .

Most Liked

Qqwy

Qqwy

TypeCheck Core Team

I might have a go at it myself, but as I already have ten different packages on Hex that I try to maintain and on the side make long days at the university and at work, I expect to have little time to do so in the near future. So if someone else wants to hack away on this: please go ahead! :smiley:

DianaOlympos

DianaOlympos

I would love to look at it, but i am a bit busy too. Additionaly, to really use Patrizia trees properly, i am affraid we would need to go down into NIFs. Otherwise you would need to use binaries to implement them.

Or i am missing something.

h4cc

h4cc

Porting a solved problem you already understand is a wonderful exercise to learn more about elixir.

Having a goal like such a library available on hex afterwards will always be a benefit in the future.

How about you porting it to elixir? :slight_smile:

Last Post!

Qqwy

Qqwy

TypeCheck Core Team

I believe you can just use Integers and use Bitwise.band to check if the desired bit of the given node is 1 or 0, which would be faster than binaries (And hopefully the BEAM can optimize this to use word-sized ints).

Where Next?

Popular in Discussions Top

AstonJ
Are there any Elixir or Erlang libraries that help with this? I’ve been thinking how streaming services like twitch have exploded recentl...
New
AstonJ
Please see the new poll here: Which code editor or IDE do you use? (Poll) (2022 Edition) It’s been a while since we first asked this, I...
208 31695 143
New
JakeBecker
TL;DR: I’ve just released an implementation of Microsoft’s IDE-independent Language Server Protocol for Elixir. It adds language support ...
1144 54921 245
New
und0ck3d
Hello everyone! A few days ago I’ve created a topic here about how people were creating CMSs with Elixir and Phoenix. I’ve been studying...
New
lorenzo
Hey everone! I created a prototype for my app using Nodejs for the api. But the framework I chose wasnt great (in general theresnt any g...
New
tmbb
This is a post to discuss the new Phoenix LiveView functionality. From Chris’s talk, it appears that they generate all HTML on the serve...
342 18584 126
New
Rustixir
Hi everyone, im working on find best language/framework/system for high concurrency, high performance and stable performance after wor...
New

Other popular topics Top

minhajuddin
I have seen a lot of code which picks the first element from a list using Enum.at(0) instead of List.first. Is there a reason why people ...
New
New
chrismccord
Phoenix 1.4.0 released Phoenix 1.4 is out! This release ships with exciting new features, most notably with HTTP2 support, improved deve...
688 31494 112
New
gausby
I asked this very same question on twitter and got some interesting feedback, but I thought it would be a good question to ask here as we...
1207 40042 209
New
alice
Hey, Just curious what are the main benefits of Elixir compared to Clojure? When is Elixir more useful than Clojure and vice versa? Th...
New
Patoshizzle
After calling mix ecto.create I get this error: 17:00:32.162 [error] GenServer #PID<0.412.0> terminating ** (Postgrex.Error) FATAL...
New

We're in Beta

About us Mission Statement