Hi all!

As you may know, it is very common in idiomatic Elixir code to work with transformations on datastructures, especially those implementing the Enumerable protocol. Usually we use the functions from Enum (and sometimes from Stream) to manipulate these, often in a pipeline of multiple steps.

Since the Enumerable protocol is an implementation of the Foldable concept from category theory, whose main fundamental operation is reduce(in other languages also known as ‘fold’), which always outputs a list, we end up using lists virtually everywhere.

This means that in an Enum pipeline, a lot of intermediate lists are being generated. I seem to remember José explaining on the mailinglist back in the day(note: I was unable to find it; if you know where this was mentioned, let me know and I’ll link to it!) that Enum’s functions were intentionally not implemented as macros to make it easier to follow stack traces when something broke.

I think this is definitely the right choice, especially since a lot of Elixir code is written with “IO-bound” operations in mind, in which sheer computing speed is less important.

However, it did start making me wonder: What about creating a FastEnum drop-in replacement, where map, reduce, etc. would be implemented as macros that would fuse consecutive operations together to improve performance?
In many cases, a pipeline of Enum-functions could be transformed into a single for-comprehension. Besides the added benefit of fusing consecutive calls, for is also extremely well optimized by the BEAM.

Now, why did I start this topic? I essentially have two questions/topics for discussion:

• Do you think a library like this would be worthwhile?
• Do you happen to know whether someone already performed any exploratory work in this direction? (The ideas presented here are, after all, far from novel.)