I Challenge you: Implement containers!

Yes, although there is also something to be said for keeping each of the packages small (only containing the public API and one or two default implementations), with a bunch of other implementation packages to be used when more efficiency or other functionality is required.

If this will not be part of the standard library some day, it would be excellent to have all of these containers in one package. If you have one package that supports a large set of standard data structures then it will be much easier for someone working on a BLAS library to make use of it. Having just a couple of well-thought-out packages that contain most basic data structures and the most common math functions would be tremendously useful, in my opinion. You can then tell people, “Just use the math and containers packages”…

Okasaki has received an overhaul and is now released as v1.0!

Changes are:

• Everything is now documented!
• ErlangQueue and ErlangDeque implementations that wrap :queue.
• Implementing FunLand.Mappable for the queues and deques.
• More tests!

That would be nice, but if doing that I’d link to a BLAS-like library instead (think NumPy for Python, but for Erlang). That way you can built up a set of instructions (macro?) that is sent to the BLAS engine to process potentially huge amounts of data at once then return the result (probably not explicitly returned either but rather as a BEAM resource ID so you can use the API to pull out bits of it or to an erlang format but keeping the base of it as highly optimized as possible, again similar to how Python does it).

With the new async: on option by default in OTP20 this would fit that use-case so well, and I would definitely use such a library. Honestly I really would build it on top of a BLAS library (one that does not crash please if you link it as a NIF or so).

Also, you’ve been busy! ^.^

Inspired by the earlier discussion with @rvirding, I am currently creating a library, Iter, that allows to iterate over arbitrary data structures in a possibly non-destructive way.

That is, where Extractable.extract will extract an item and return a collection with that item removed, Iter.next() will return the next item and a collection that might or might not be altered.

This means that Iterator structs wrapping things like ETS tables, :gb_sets and :gb_trees can be made as well.

There are two kinds of Iterators inside Iter:

• ‘Normal’ iterators which might convert the thing they iterate on to a structure more appropriate for iteration. (This is used for many datatypes that right now do not have built-in first/1, next/2-support; they are frequently changed into lists)
• and ‘Persistent’ iterators which are guaranteed to be able to reconstruct the thing they iterate over at the end.

Note that the functionality of a PersistentIterator is a superset of a normal Iterator, possibly at the cost of some time and memory efficiency.

I am not yet sure what (if any) advantage a PersistentIterator has over a non-persistent iterator, as we usually still have access to the original data structure before it was turned into an iterator.

But I wanted to build it, in part because I thought of a fun way to (ab)use improper lists to reverse-concatenate two lists (restoring a zipper back to its original list): elixir-iter/lib/iter/implementations/list.ex at master · Qqwy/elixir-iter · GitHub

And in part because there very well might be a proper use for them.

Because these Iterators only iterate on-demand, they are per definition lazy.
I am going to think about what would be the best way to create higher-order iterators (‘iteratees’ or ‘iterator adaptors’), and how the efficiency of the thing could be improved.

This style of iterating is based on the ‘Iteratee’ concept used in Haskell, and also the Iterators in Rust. (Rusts’ iterators work with owned, mutable data however, and therefore are both faster and guaranteed to extract items returned by the iteration from their containers; two guarantees Iter cannot give you).

And just after I said that, I decided to finish up Okasaki as well.
It still needs some more documentation, but it definitely is stable enough to release on Hex.PM as-is.

And a final one for today: Arrays.

Similar to Prioqueue, Okasaki and Sets, this is a common interface to work with arrays that have fast random element access.

It contains two implementations right now, namely a version that wraps Erlang’s :array, and MapArray that uses integer keys as indexing into a map (But in a somewhat different way than I do it in Tensor; MapArray is not a sparse but a terse representation).

And… Sets is released!

Similar to Prioqueue and Okasaki, this is a common interface to work with Sets, with multiple implementations that can be chosen between either when creating the set, or in your application’s documentation.

Implementations for MapSet, :sets, :gb_sets and :ordsets are included.

Oh, before I forget!

Extractable and Insertable have been updated to 0.2.0 as well. This is a backwards-incompatible change. (But probably the last; I think the API is now stable and if this is indeed the case, soon a stable 1.0 version will be released).

The backwards-incompatible change is that in case of errors, instead of returning :error, the implementations should return {:error, reason}, with standardized reasons for Extractable being :empty, and for Insertable being :invalid_item_type and :full.

Prioqueue has been updated to reflect these changes as well, now being version 0.2.4 .

All right!

Tensor has been updated to version 2.0. This version is backwards incompatible because of multiple major changes:

• All modules are moved under the single Tensor namespace, to follow HexPM package publishing rules. use use Tensor to alias all of Tensor, Matrix and Vector in your code and us them just as before.
• Numbers has had multiple important changes between version 1.0 and 4.0, these changes have been reflected in Tensor.
  Other, new features are:
• FunLand.Mappable implementation, FunLand.Reducable implementation.
• Extractable implementation.
• Insertable implementation.
• Tests for the above.