Introducing `for-let` and `for-reduce`

[stefanchrobot]

@fuelen

> for a = 1, b <- [1, 2, 3], do: b
** (CompileError) iex:4: for comprehensions must start with a generator

Try this:

for a <- [1, 2, 3], b = 1, do: {b, a}

That’s why I prefer an explicit let (or a variant).

[dimitarvp]

@massimo

I understand, but It actually doesn’t, in plain English.

Most people in the world, me included, are not native English speakers. I’ll have to go reach for advanced English courses to understand why does “where” make sense in this context. Don’t introduce friction where it doesn’t need to be.

@massimo

in Haskell where is bound to a surrounding syntactic construct and it’s used to bind local variables but it’s not an expression, while let is.

Like OCaml’s recursive let var = ... in construct? Also I am not sold on bringing Haskell-like features in Elixir but hey.

Furthermore, to me it still looks confusing and I’ll have to read the new tutorial in detail, likely twice, before I understand what this syntax is supposed to achieve. Not a good sign, sorry.

@benwilson512

Meta point: Given that 95% of the commentary here is now (totally valid) bike shedding about let vs other words

Strong disagree. What you call “bikeshedding” here is a legitimate discussion about how confusing or intuitive such a syntax is. Hand-waving it away as bikeshedding is disrespectful and doesn’t help.

Elixir is known to be quick and easy to grok. People, me included, are concerned that such a proposal goes against this perception. You are free to ignore it and push through but the goal of the discussion is to make you aware of how others think about it.

@benwilson512

I think the overall proposal is a success

Weird statement for people who haven’t been in the loop, mind elaborating? What’s the true benefit that this brings in the life of a everyday programmer?

[fuelen]

@stefanchrobot

That’s why I prefer an explicit let (or a variant).

having = at the beginning is an explicit expression as well

[LostKobrakai]

I’ll for sure need to look at this some more, but my first reaction was that I’d actually prefer the syntax of the :reduce option written in it’s current form. That one is for sure less declarative and with less helpful errors, but does look less foreign and makes it explicit that a reduction is happening by using acc -> result in the block. But there’s no denying that for is great compared to their Enum counterparts.

[josevalim]

@fuelen

Can’t we simply detect operation by the first argument passed to for? Like, if = is present in expression at the beginning, then treat it as accumulator for reduce operation.

Unfortunately no because there is ambiguity with filters. For example, I can use x = expr as a filter and if expr is false or nil, it doesn’t continue. Does it mean that I can’t ever initialize an state as false or nil? Plus let makes it clearer it is changing the return type.

@dimitarvp

Furthermore, to me it still looks confusing and I’ll have to read the new tutorial in detail, likely twice, before I understand what this syntax is supposed to achieve. Not a good sign, sorry.

I recommend doing the exercises in the guide using their Enum equivalent or recursion and compare them side by side. There are a lot of questions if this is worth it but unless you try to solve the problem, it is easy to see the solutions as complex. Feel free to start a new thread with solutions so there is no spoilers for people reading this one.

[massimo]

@stefanchrobot

Regarding where - as noted by @benwilson512, it already has a specific meaning in Elixir - it tests a condition.

that’s true only for Ecto, that maps on SQL, it has no meaning in Elixir or in Erlang.

@dimitarvp

Most people in the world, me included, are not native English speakers. I’ll have to go reach for advanced English courses to understand why does “where” make sense in this context. Don’t introduce friction where it doesn’t need to be.

I am not either.

I learned about where at school, studying business English, and only in SQL means only if.

In Ruby it’s select.

@dimitarvp

Like OCaml’s recursive let var = ... in construct? Also I am not sold on bringing Haskell-like features in Elixir but hey

let ... in exists in Haskell too, but you lose the information of the comprehension that in Elixir is for.

I quoted Haskell documentation not because it’s from Haskell, but because it makes sense to me that if something is specific to a block, you either declare the variables at the beginning, Pascal like

var
	x: integer value 42;

# for var isn't half bad now that I think about it 

or at the end to mean this is valid only where the block scope exists

I like the idea, but for-let is backwards compared to every other Elixir idiom

with starts with a matching clause, which is all it is about,for starts with a generator, which is all it is about, for-let starts with declaring an accumulator, which is not really intuitive IMO because for-let is a map/reduce in disguise.

[josevalim]

@massimo

for-let starts with declaring an accumulator, which is not really intuitive IMO because for-let is a map/reduce in disguise.

One of the first things you do on reduce/map_reduce is to declare the accumulator.

[stefanchrobot]

@massimo

@stefanchrobot

Regarding where - as noted by @benwilson512, it already has a specific meaning in Elixir - it tests a condition.

that’s true only for Ecto, that maps on SQL, it has no meaning in Elixir or in Erlang.

You’re right. I confused it with when. But I take it as an argument against using where.

[thiagomajesk]

@josevalim

I want to be very clear it is not only about making Elixir more approachable. The point is that it is both more approachable and often more elegant than the options currently available in Elixir.

I see, I was only taking into consideration the post’s preamble:

The goal is to show how for can be useful to solve several problems in a format that developers may be familiar with, while still building an intuition on functional ideas.

---

@josevalim

Here is a simple question, can you write the example below using Enum? If so, how? And, once you do, which solution do you prefer?

@stefanchrobot

You don’t actually need for, but learning it - including the newest additions - will make your code better. I’m really looking forward to this and even more to a way to early-exit the comprehension so that I can get rid of all the not-so-pretty Enum.reduce_while calls.

I think that perhaps you missed my point, I do know how useful for is for those cases that are already supported by the language and I’m not against it whatsoever. I’m only worried that let and reduce are introducing a specific new syntax that it’s only applicable to a very specific scenario - such that it needs a completely new concept of ‘qualifiers’.

I think that the benefits to making it “cleaner” as @josevalim has put it are completely valid, but I don’t think it answers the “at what cost” part… To me at least (at first sight), it seems that the cost we are paying is to introduce a kind of esoteric new syntax that it’s very very specific to one use-case, which feels weird to me.

@josevalim

Therefore, a good way for you to answer the question of “at what cost?” is for you to go through each example in the guide and try to write them without using for, using a single function in Enum or recursion. If you can write the Enum variants and if you think they all look better, then this is probably not worth it. But what will most likely happen is that you will find the for variants to be cleaner.

Do you have a specific example that you personally like that displays the true power of let and reduce over Enum? I started doing this exercise and it seems that mostly what I’m avoiding is storing the computed values in an accumulator, which is not that cleaner (not that worse either). However, it still seems more natural to me (less magic).

I kind of understand the appeal of having a new way to expand and make for a little more powerful, but I’m still not sold on the new syntax though. I agree with @joaoevangelista on the subject that the concept of qualifiers is not present in other parts of the language, which is my main reason for having this weird look at it so far.

I’d be frying my brain a little less if the new syntax used a more familiar concept that we see everywhere, like passing options to functions as arguments (I know the reasoning is explained in the proposal) or if let, reduce and async were plain functions that make sense to use in other scenarios (lego mentality). I know that in the end (it doesn’t even matter jk), it’s all about the tradeoffs.

[mat-hek]

@josevalim

Error messages

By declaring the shape we want to return in let/reduce, we can provide really good error messages. For example, imagine the user makes this error:

iex> for let {sum, count} = {0, 0}, i <- [1, 2, 3] do
...>   sum = sum + i
...>   count = count + 1
...>   {i * 2, sum}
...> end

The error message could say:

** (ComprehensionError) expected do-end block to return {output, {sum, count}}, got: {2, 1}

Let’s assume that the sum is a complex number and therefore it’s a 2-tuple itself Could we still catch that?

[dimitarvp]

@josevalim

There are a lot of questions if this is worth it but unless you try to solve the problem, it is easy to see the solutions as complex.

for is super useful and I’ve used it many times and I love it. No argument there. The new syntax’s value is still unclear to me however, but I’ll do a few exercises and that should bring it to light – as you suggested.

@josevalim

Feel free to start a new thread with solutions so there is no spoilers for people reading this one.

Apologies, I have no clue what you are saying. We aren’t discussing movies that are still in the cinema here.

[josevalim]

@thiagomajesk

I see, I was only taking into consideration the post’s preamble:

I see. At some point the preamble said something how for can be useful for both those learning the language and power users alike but it seems that was lost during the reviews.

EDIT: it was moved to the conclusion.

@thiagomajesk

such that it needs a completely new concept of ‘qualifiers’.
[…]
a kind of esoteric new syntax

I think you are too focused on the concept of qualifiers and perhaps this is a sign we should not be using this term at all, as it isn’t really anything new and no new language capabilities are necessary to make this work. For example, StreamData is using a similar construct (check all) for years.

@thiagomajesk

Do you have a specific example that you personally like that displays the true power of let and reduce over Enum?

Try this one: https://github.com/josevalim/nested-data-structure-traversal It is one of the issues that sparked the discussion. The discussion in the elixir-lang-core mailing list has an example of using for-let to solve it (the second email from Ben). You can compare it with the examples in the Elixir folder.

[josevalim]

@dimitarvp

Apologies, I have no clue what you are saying. We aren’t discussing movies that are still in the cinema here.

I am asking to not post the Enum versions in this thread because maybe others want to do the same exercise as you. Seeing your solutions you “spoil” for those who want to do the same.

@mat-hek

Let’s assume that the sum is a complex number and therefore it’s a 2-tuple itself Could we still catch that?

Unfortunately no.

[josevalim]

Thanks everyone for the feedback so far. I think a couple things were lost in translation over the several discussions that have happened in the mailing list so I will close and unlist this topic for now. I will resubmit it in the next hours with more context and less discussion points. Stay tuned!