Someone mentioned “local scoped variables” and it actually sounds good… The $
or @@
syntax to differentiate those variables while keeping the usage of =
sounds interesting as well (looks better without the let
though).
Editors can help with readability without adding extra syntax to the variable. Rider adds an underline for mutable variables in F#
I have re-read the past thread with the let
proposal. What I proposed is not the same, the major difference being the variables bound in let
need to be returned as the tuple from the block.
I didn’t have a problem with for let
but I didn’t like that proposal because the return tuple from the comprehension is not connected to the let binding in the next round in an obvious or intuitive way. Needing to provide a result tuple with the results ordered in a particular way and the usage of let
didn’t convey mutation/rebinding on each iteration. It would be better conveyed with say var
instead of let
.
I am suggesting similarly declaring the state in the comprehension (let’s not worry about syntax for now) but automatically using whatever value they bind to for the next “iteration”. If that state is also to be returned as a result of the comprehension (i.e. binding to variables in the parent scope), it similarly will take the current binding of whatever the comprehension set or left them at.
This way the comprehension state is continuously being reused and rebound as needed each iteration without manual effort to explicitly return it. It’s basically like mut
within the comprehension and some way to return this state to the outer scope at the end which is not part of the for
map/reduce result. You could return a tuple containing the result and the state (not as obvious IMO) or provide some other way to express the binding between the comprehension state values and the parent scope.
This was certainly discussed at some point, so perhaps it was in a past-past proposal or in a branched thread?
In any case, it is worth pointing out that tying it to comprehensions in your example is purely a syntactic restriction. The implementation complexity would be pretty much the same and anyone could get the same behaviour as this proposal by doing:
for _ <- 1..1, ^session_counter, ^lesson_counter do
for lesson <- lessons do
...
end
end
So tying it to comprehensions only makes the comprehensions feel more bloated (beyond generators, filters, into, and reduce), which was one of the main concerns before, and less likely to be used.
I fear that the mental model for the code execution will be suddenly way harder because I can see less experienced developers (at least reagarding FP patterns) abusing those mutable variables because they think in a mutable way by default.
And it will be hard to argue with them that the code is a mess because yes, there is more whitespace delimitations, less state carrying, etc. So obviously the code is easier to understand.
Except it’s not. Now whenever you have a 15-lines functions you will have to constantly check if the variable is mutable whenever you see =
. A different operator or assignment form has limitations, (cannot mix mut and regular variables in assignment) but at least it would limit those problems.
I fear there will be many cases of assignment where the semantics would not be clear:
mut x = 10
x =
case {:a, 1} do
{:a, x} -> x + 1
{:b, x} -> x * 2
end
Or the compiler will forbid a lot of stuff and get in the way.
So I would say no to that. But I like the proposal anyway, I’m glad this would be possible in Elixir with some compiler magic.
I feel that the first step towards the section/lesson problem should be smaller. For insance turn this:
vars = []
vars = if opts[:foo], do: [{:foo, get_foo()} | vars], else: vars
vars = if opts[:bar], do: [{:bar, get_bar()} | vars], else: vars
into this:
vars = []
vars ^= if opts[:foo], do: [{:foo, get_foo()} | vars]
vars ^= if opts[:bar], do: [{:bar, get_bar()} | vars]
I do take your point that it’s a lot of special tratment for a single construct rather than a broader language capability.
Comprehensions do appear to have something about them that, pardon my pun, “reduces” their use in actual code vs other idioms.
I can certainly see where comprehensions are useful for flattening the processing with multiple generators. My feeling is that comprehensions don’t compose with pipelines so we just use Enum or Stream which does.
Perhaps an Enum.for
that works similar to for
where the generator input goes on the left and can have additional generators and state would cross the cavern, but I’m just hypothesizing.
- What should we call them? Local state? Local accumulators?
Calling them local accumulators makes the concept a lot easier to understand imho.
It fits the reason for proposing this feature in the first place (to locally accumulate some values).
The moment state or mutability is mentioned it gets confusing. Since you can also keep track of state in Elixir via processes. And as you mentioned the compiled code is still immutable.
Ok, now i understand that it’s about not losing the possibility of using the “accumulators” in all the mechanisms that comprehensions offers. However, opening the accumulators outside of the comprehensions seems to open the door to new (or old) strange code
I am sure i will start reading code like
let $flag = false
if $flag == false do
$flag = true
end
....nested block...
$flag = false
and lost the restrictions that allows to think in a different way.
what about a new kind of comprehensions, maybe forlet
or some other that limits the scope of accumulators only to the concept of comprehensions?
Yes! Separating the concept from plain variables is imperative and it makes a lot of sense… In this case, I think the possible @@
syntax mentioned by @josevalim would fit the mental model very well. IMHO because we already use @
to define attributes (which can be used somewhat like compile-time accumulators) it makes the mental jump a lot easier and avoids confusion.
I must say that I like this version a lot…
@@list = []
if some_value? do
@@list = ["prefix" | list]
end
And I find its explicitness very appealing…
@@section_counter = 0
@@lesson_counter = 0
for section <- sections do
if section["reset_lesson_position"] do
@@lesson_counter = 0
end
@@section_counter = @@section_counter + 1
lessons =
for lesson <- section["lessons"] do
@@lesson_counter = @@lesson_counter + 1
Map.put(lesson, "position", @@lesson_counter)
end
section
|> Map.put("lessons", lessons)
|> Map.put("position", @@section_counter)
end
I completely agree with this as well. I think it also helps the mental model around “what happens when I pass this to a function”? In both @ and @@ the answer is the same: you just get a regular value. They aren’t defining special values, just special syntax. From the perspective of a called function everything is as normal.
Why cannot solve this “problem” using some macro magic?
It’s actually easy to do this withing a scoped block using macros defined in a 140-line file. I’ve uploaded a gist here: dark_mutable.ex · GitHub (I can’t push it to the main github repo because right now I’m on a network that doesn’t suppor SSH).
You can see it being used in a simple factorial function (note that mutable assignment is done using the <<~
operator; I don’t think that mutable variables should be assigned with the =
operator, even though it’s very easy to implement it):
defmodule DarkKernel.DarkMutableTest do
use ExUnit.Case, async: true
import DarkKernel.DarkMutable
def factorial(n) do
# Define a mutable variable, which is only accessible inside the block
mutable! result: 1 do
for i <- 1..n do
# Update the variable in a way that's propagaded "up" the AST scopes
# in the entire `mutable!` block.
result <<~ result * i
end
# Return the "mutable variable"
result
end
end
test "factorial function works" do
assert factorial(1) == 1
assert factorial(2) == 2
assert factorial(3) == 6
assert factorial(4) == 24
assert factorial(5) == 120
end
end
This generates the following code:
18:07:07.305 [debug] test/dark_kernel/dark_mutable_test.exs:7
┌────────────────────────────────────────────────────────────────────────────────────────────────────
│ ⤷ Code:
├╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶
│ for i <- 1..n do
│ DarkKernel.DarkMutable.__set__(result, DarkKernel.DarkMutable.__get__(result) * i)
│ end
│
│ DarkKernel.DarkMutable.__get__(result)
├────────────────────────────────────────────────────────────────────────────────────────────────────
│ ⤷ Generated code:
├╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶
│ result = make_ref()
│
│ try do
│ DarkKernel.DarkMutable.__set__(result, 1)
│
│ (
│ for i <- 1..n do
│ DarkKernel.DarkMutable.__set__(result, DarkKernel.DarkMutable.__get__(result) * i)
│ end
│
│ DarkKernel.DarkMutable.__get__(result)
│ )
│ after
│ DarkKernel.DarkMutable.__delete__(result)
│ end
├────────────────────────────────────────────────────────────────────────────────────────────────────
The code above doesn’t generate the
@josevalim, given that I have implemented this in about 1h30min, I don’t think major language changes are needed, and I don’t see much use for the mut
keyword.
I’d even argue that having the variables scoped inside a clearly delimited block is a much simpler solution. And this implementation automatically gaurantees that external processes can’t change the references, because there isn’t a way of sending the reference somewhere else. Reference values are always passed by copy.
As I see it, doing this via the process dictionary is not “implementing this”. A key part of the proposal is that it would be syntactic sugar over normal immutable operations. Solving it with actual mutability is a significant difference.
Ok, so that part is important to you. Ok, in that case my solution is defiinitely unacceptable. Also, my solutino also deviates regarding this:
I my implementation, the code above works:
def factorial_with_map(n) do
mutable! result: 1 do
# Update the product inside a map function
Enum.map(1..n, fn i -> result <<~ result * i end)
# Return the mutable variable
result
end
end
and in my opinion it should work. If we want mutable variables, then they better be real mutable variables instead of a very artificial thing that works in for loops and not in map functions. But that is just my personal opinion, of course.
Personally, having made the transition to thinking in functional terms, I no longer ever wish I could “borrow” the mutable style just to make a some code more readable. Instead I reach for other refactoring tools, clearer variables, breaking out into helper functions etc. I think if ‘local mutability’ like this existed I would just not use it and argue for style guidelines in my projects to avoid it because in some cases I think it’s better to just not even have to make the choice (both when writing and interpreting code). But I have been called a curmudgeon so…
…that said, I like the @@
var syntax, especially in light of how attributes can accumulate, as others have observed.
I’m doing my best to help that!
I’m coming to this thread late, but I’m glad I read to the bottom before posting since the process dictionary has already been mentioned. Given that we already can implement the desired code with pdict, the concern here is to only appear to be mutable while actually outputting immutable beam code. I’m not convinced the value is worth the cost.
The value is to be more readable in those circumstances which can use the syntax, which doesn’t seem very often. The cost is one more special case to consider. I’ve been with Elixir since beta, so learning one more thing incrementally is not a problem, but it is one more barrier to newcomers.
Thank you. These points convinced me that @@
is a very sensible prefix in case we want to go ahead with this proposal. The “a” in @ (“at”) also matches the “a” in accumulator (as it matches the “a” in attributes). I think I have all of the pieces in place to submit a new proposal, so I will do that later. Thank you.
As you pursue this, I’d be interested in seeing an analysis of how @@var
would contrast with existing @var
semantics:
-
Assignment
Module attributes assign via
@var value
. It looks like accumulating attributes will use@@var = value
, is this a vector for confusion? -
Assignment scoping
Module attributes are (usually) assigned to in module bodies. It looks like accumulating attributes are intended to be scoped to function bodies. What will the devex be like if you try to define or assign them them at the module level? Is there a vector for confusion there? Is there a sensible functionality for them at that level, like an accumulation useful in
@on_def
callbacks? Would that sensible functionality cause more confusion if implemented?
I don’t think it’s a good prefix at all! Currently, the “syntax” @@x
is valid and parses into this:
iex(1)> quote do @@a end
{:@, [context: Elixir, imports: [{1, Kernel}]],
[
{:@, [context: Elixir, imports: [{1, Kernel}]],
[{:a, [context: Elixir], Elixir}]}
]}
So it wouldn’t be “real syntax” (I understand that “real syntax” is not something that’s easy to apply to a language like Elixir), it would be just applying @
to a module attribute (not exactly, I know that @ can be rewritten into eveyrthing, but still, the point stands). Wouldn’t it be better to represent mutability by using an “arrow operator” (such as <~
or <<~
for example) and leaving the variable name as it is?
It’d be real syntax with a trivial change in the parser to introduce a new high-precedence operator.
It parses into AST, but that AST cannot be evaluated anywhere in a program without raising an error, so is not currently valid syntax.
With the caveat that what follows depends on your definition of “syntax” (which is quite flexible in how the concept is used in Elixir), @@x
is definitely valid syntax and can be incorporated into real programs!
defmodule DarkKernel.StupidAtTest do
use ExUnit.Case, async: true
import Kernel, except: [@: 1]
defmacro @(_arg) do
quote do
"STUPID"
end
end
test "stupid result is stupid" do
a = 1
# Stupid is stupid
assert @a == "STUPID"
# More stupid doesn't actually make it more stupid
assert @@a == "STUPID"
# Even more stupid, but still the same
assert @@@a == "STUPID"
end
end
Turning @@x
into a new operator could work, but it would definitely be a backward-incompatible change.