Qqwy

Qqwy

TypeCheck Core Team

I Challenge you: Implement containers!

tl;dr: I challenge you: let us implement containers for Elixir, and use this topic to coordinate the effort.


Hello, dear fellow Elixir developers.

Recently, multiple discussions (like this and this) have popped up about ‘Elixir not having a built-in data type XXX’.

While for some part we can rely on things that are already part of Erlang (such as :queue or :array), there are of course drawbacks to using Erlang modules:

  1. No Protocol support.
  2. No documentation (that is part of IEx).
  3. No introspection, as they do not work on structs (structs are, after all, an Elixir invention).
  4. Most of these libraries were made before Maps were a thing (Maps were introduced in OTP 17), and using maps we might create more efficient data structures (in space/time) in some cases.

On the other hand, the Elixir standard library only contains the container types Lists, Maps, and MapSets (and Range, but it is clearly the ‘odd one out’). It definitely is not the role of the Elixir standard library to provide built-in datatypes for everything, however. That is something where FOSS libraries can shine.

I myself have until now built Tensor, which introduces sparse vectors, matrices and n-dimensional tensors (collections that allow for amortized constant-time element access/update as they are built on top of maps).
I am currently in the process of creating Okasaki, which is a library containing multiple different versions of Queues and Deques (double-ended queues) with different time/memory properties.


One thing I realized, which is partly why I am starting this topic, is that the Enum and the Enumerable protocol will throw away the structure of the container you had, always returning lists as result.
I therefore am going to implement a common interface, based on the actual properties that are required to perform certain operations.

I challenge you to start implementing container data structures and release them as FOSS libraries. I think it would be wonderful to start a semi-coordinated effort through this forum, to ensure that Elixir gets a nice zoo of data structures that can be used to solve a wide variety of problems.


Suggested Container datatypes (And what libraries are working on them)

(feel free to suggest one)

  • Trees: (Many other container types can be built on top of trees)
    • Binary trees
    • Red-Black trees
    • Rose trees
    • 2-3 finger trees
    • Patricia trees
  • Queues: Okasaki tries to implement multiple variants of these.
    • FIFO Queues
    • LIFO Stacks (List covers this somewhat, but maybe there are other ways to implement stacks with different guarantees as well? (And then have a unified interface for any kind of stack-like implementation? :heart_exclamation:)
    • Deques (Double-Ended Queues)
  • Priority Queues Prioqueue
  • Heaps
  • Sets: Implemented in Sets
    • HashSet (Builtin)
    • Tree-based Set
  • Multisets and Bags
  • Fast Random-Access Sequences
    • Arrays implements some of these:
      • MapArray
      • :array (heap-based array)
    • Something akin to Haskell’s Data.Sequence that allows O(1) access to both extremes of the container.
    • Immutable arrays (á la Haskell IArray? or Erlang’s :array?)
    • Someting working with DiffArrays?
  • Sparse Vectors/Matrices/Tensors: Tensor
  • Graphs
    • Adjacency list
    • Adjacency matrix
    • Incidency matrix.

Most Liked

Qqwy

Qqwy

TypeCheck Core Team

Okasaki has received an overhaul and is now released as v1.0!

Changes are:

  • Everything is now documented! :sunglasses:
  • ErlangQueue and ErlangDeque implementations that wrap :queue.
  • Implementing FunLand.Mappable for the queues and deques.
  • More tests!
NobbZ

NobbZ

I think it does NOT make sense to put everything into a single package. Separation of concerns and stuff :wink:

I’m not sure though, how to organize packages, dependencies, structures and algorythms.

rvirding

rvirding

Creator of Erlang

Not really. One problem with your example is that you are updating the map every time which is inefficient. I was thinking something along the lines of how ETS does it with first/next:

10> {Key1,Val1} = maps:first(Map).
11> {Key2,Val2| = maps:next(Key1, Map).
12> {Key3,Val3} = maps:next(Key2, Map).
13> error = maps:next(Key3, Map).

(in Erlang) Definitely possible. ETS does it like this but only returns the key, in this case returning {Key,Value} is much better. I implemented this in my 2-3 trees as I needed it for Luerl.

Where Next?

Popular in Discussions Top

Jayshua
I recently came across the javascript library htmx. It reminded me a lot of liveview so I thought the community here might be interested....
New
pillaiindu
In django there is a cache framework backed by memcached. Rails also puts a lot of emphasis on caching, and even the idea of russian-doll...
New
WolfDan
After doing a port from a c++ library to my project in phoenix I’ve seen that I need a faster way to run this algorithm and I found this ...
New
mmport80
I have put far too much effort into Dialyzer over the last year or so - and basically - I doubt it’s worth the effort. It’s not as easy ...
New
arpan
Hello everyone :wave: Today I am very excited to announce a project that I have been working on for almost 3 months now. The project is...
New
lucaong
Hello Elixir and Nerves community, I have been working for a while on an open-source embedded key-value database for Elixir, that I call...
230 13924 124
New
gausby
I asked this very same question on twitter and got some interesting feedback, but I thought it would be a good question to ask here as we...
1207 39297 209
New
CharlesO
Erlang :list.nth simple, but 1 - based nth(1, [H|_]) -> H; nth(N, [_|T]) when N > 1 -> nth(N - 1, T). Elixir Enum.at … coo...
New
fireproofsocks
I’ve been working on an Elixir project that has required a lot of scripting. I usually reach for Elixir because I like it more (and in th...
New
saif
Hello everyone, Long time lurker first time poster here. I’ve recently begun working on Elixir full-time again! :raised_hands: It’s been...
New

Other popular topics Top

danschultzer
None of the current solutions worked well for me, so I went ahead and built a user management system from scratch. This project took far...
548 29377 241
New
vertexbuffer
Hello, can anybody help here..? I have a list of players and I what to delete an element, but every for loop the list is reverting to ori...
New
chrismccord
As promised, the first release candidate of Phoenix 1.3.0 is out! This release focuses on code generators with improved project structure...
New
Patoshizzle
After calling mix ecto.create I get this error: 17:00:32.162 [error] GenServer #PID<0.412.0> terminating ** (Postgrex.Error) FATAL...
New
JeremM34
Hello, how can I check the Phoenix version ? Thanks !
New
josevalim
Hi everyone, One of the features added to Elixir early on to help integration with Erlang code was the idea of overridable function defi...
New
vonH
When I run the Plug and I recompile I wind up having to use Ctrl C to quit iex and start again. Witht the help of rlwrap I can use the cu...
New
axelson
This post is a wiki (feel free to hit the edit button near the bottom right of this post to add your own changes!) This post collects co...
239 47930 226
New
Qqwy
Update: How to use the Blogs & Podcasts section You can post links to your blog posts or podcasts either in one of the Official Blog...
3271 126479 1222
New
hariharasudhan94
Lets say I have map like this fetching from my database %{"_id" => #BSON.ObjectId<58eb1a7a9ad169198c3dXXXX>, "email" => ...
New

We're in Beta

About us Mission Statement