Qqwy

Qqwy

TypeCheck Core Team

I Challenge you: Implement containers!

tl;dr: I challenge you: let us implement containers for Elixir, and use this topic to coordinate the effort.


Hello, dear fellow Elixir developers.

Recently, multiple discussions (like this and this) have popped up about ‘Elixir not having a built-in data type XXX’.

While for some part we can rely on things that are already part of Erlang (such as :queue or :array), there are of course drawbacks to using Erlang modules:

  1. No Protocol support.
  2. No documentation (that is part of IEx).
  3. No introspection, as they do not work on structs (structs are, after all, an Elixir invention).
  4. Most of these libraries were made before Maps were a thing (Maps were introduced in OTP 17), and using maps we might create more efficient data structures (in space/time) in some cases.

On the other hand, the Elixir standard library only contains the container types Lists, Maps, and MapSets (and Range, but it is clearly the ‘odd one out’). It definitely is not the role of the Elixir standard library to provide built-in datatypes for everything, however. That is something where FOSS libraries can shine.

I myself have until now built Tensor, which introduces sparse vectors, matrices and n-dimensional tensors (collections that allow for amortized constant-time element access/update as they are built on top of maps).
I am currently in the process of creating Okasaki, which is a library containing multiple different versions of Queues and Deques (double-ended queues) with different time/memory properties.


One thing I realized, which is partly why I am starting this topic, is that the Enum and the Enumerable protocol will throw away the structure of the container you had, always returning lists as result.
I therefore am going to implement a common interface, based on the actual properties that are required to perform certain operations.

I challenge you to start implementing container data structures and release them as FOSS libraries. I think it would be wonderful to start a semi-coordinated effort through this forum, to ensure that Elixir gets a nice zoo of data structures that can be used to solve a wide variety of problems.


Suggested Container datatypes (And what libraries are working on them)

(feel free to suggest one)

  • Trees: (Many other container types can be built on top of trees)
    • Binary trees
    • Red-Black trees
    • Rose trees
    • 2-3 finger trees
    • Patricia trees
  • Queues: Okasaki tries to implement multiple variants of these.
    • FIFO Queues
    • LIFO Stacks (List covers this somewhat, but maybe there are other ways to implement stacks with different guarantees as well? (And then have a unified interface for any kind of stack-like implementation? :heart_exclamation:)
    • Deques (Double-Ended Queues)
  • Priority Queues Prioqueue
  • Heaps
  • Sets: Implemented in Sets
    • HashSet (Builtin)
    • Tree-based Set
  • Multisets and Bags
  • Fast Random-Access Sequences
    • Arrays implements some of these:
      • MapArray
      • :array (heap-based array)
    • Something akin to Haskell’s Data.Sequence that allows O(1) access to both extremes of the container.
    • Immutable arrays (á la Haskell IArray? or Erlang’s :array?)
    • Someting working with DiffArrays?
  • Sparse Vectors/Matrices/Tensors: Tensor
  • Graphs
    • Adjacency list
    • Adjacency matrix
    • Incidency matrix.

First Post!

NobbZ

NobbZ

There has been some effort put into exads, which at leasts interescted with your container proposal. Its a pity though that the effort stopped abprubtly, may be we cann pull something out of that?

Most Liked

Qqwy

Qqwy

TypeCheck Core Team

Okasaki has received an overhaul and is now released as v1.0!

Changes are:

  • Everything is now documented! :sunglasses:
  • ErlangQueue and ErlangDeque implementations that wrap :queue.
  • Implementing FunLand.Mappable for the queues and deques.
  • More tests!
NobbZ

NobbZ

I think it does NOT make sense to put everything into a single package. Separation of concerns and stuff :wink:

I’m not sure though, how to organize packages, dependencies, structures and algorythms.

rvirding

rvirding

Creator of Erlang

Not really. One problem with your example is that you are updating the map every time which is inefficient. I was thinking something along the lines of how ETS does it with first/next:

10> {Key1,Val1} = maps:first(Map).
11> {Key2,Val2| = maps:next(Key1, Map).
12> {Key3,Val3} = maps:next(Key2, Map).
13> error = maps:next(Key3, Map).

(in Erlang) Definitely possible. ETS does it like this but only returns the key, in this case returning {Key,Value} is much better. I implemented this in my 2-3 trees as I needed it for Luerl.

Last Post!

Qqwy

Qqwy

TypeCheck Core Team

Yes, although there is also something to be said for keeping each of the packages small (only containing the public API and one or two default implementations), with a bunch of other implementation packages to be used when more efficiency or other functionality is required.

Where Next?

Popular in Discussions Top

New
CharlesO
Erlang :list.nth simple, but 1 - based nth(1, [H|_]) -> H; nth(N, [_|T]) when N > 1 -> nth(N - 1, T). Elixir Enum.at … coo...
New
JakeBecker
TL;DR: I’ve just released an implementation of Microsoft’s IDE-independent Language Server Protocol for Elixir. It adds language support ...
1144 54996 245
New
chulkilee
Here are the list of HTTP client libraries/wrappers, and some thoughts on HTTP client in general. I’d like to hear from others how they w...
New
AstonJ
If so I (and hopefully others!) might have some tips for you :slight_smile: But first, please say which area you’re finding most challen...
New
nburkley
AWS re:Invent is on at the moment with some interesting announcements. One new feature in particular is the Lambda Runtime API for AWS La...
New
owaisqayum
I have a sample string sentence = "Hello, world ... 123 *** ^%&*())^% %%:>" From this string, I want to only keep the integers, ...
New

Other popular topics Top

Qqwy
Update: How to use the Blogs & Podcasts section You can post links to your blog posts or podcasts either in one of the Official Blog...
3271 130579 1222
New
electic
Hi, I am new to Elixir. I am trying to use the DateTime component to insert a date into MySQL however the there seems to be no way to fo...
New
grych
Hi folks, Few months ago I have announced the proof-of-concept of the library to manipulate the browsers DOM objects directly from Elixi...
639 54092 488
New
vonH
When I run the Plug and I recompile I wind up having to use Ctrl C to quit iex and start again. Witht the help of rlwrap I can use the cu...
New
JakeBecker
TL;DR: I’ve just released an implementation of Microsoft’s IDE-independent Language Server Protocol for Elixir. It adds language support ...
1144 54996 245
New
AstonJ
Seen any cool LiveView demos, sample apps or examples? Please post them here! :003:
New

We're in Beta

About us Mission Statement