Elixir language reference

Questions / Help
42

The Syntax Reference page that @michalmuskala posted above.

43

Yes, I misread that as a specific part of the page.

44

It is a special form, like %{}. It is not directly part of the syntax, but treated in a bit special way. Kind of a hardcoded macro, internally working pretty much like a sigil, but not exactly as them.

https://hexdocs.pm/elixir/Kernel.SpecialForms.html

45

You (the elixir community in general, not you specifically) seem to use the word syntax in a way I’ve never seen it used
 What you’re saying is akin to: “This is not syntax - It’s just transformed by the compiler into an AST in which the root is a macro that can’t be redefined by the user”. Isn’t that what syntax is? The rules used for the compiler to transform text into an AST?

46

The syntax, for me, is exclusively what is handled in the parser. Anything beyond that is not “syntax” but “language features”. How binaries work has no special treatment in the parser, so to me, it’s a language feature that uses regular Elixir syntax.

47

Yes, but who turns <<a, b, c>> into: {:<<>>, [], [{:a, [], Elixir}, {:b, [], Elixir}, {:c, [], Elixir}]}? Isn’t it the parser? (I assume it is, even though I have no idea how Elixir parses Elixir).

This is not the same as turning: def d(x), do: x into:

{:def, [context: Elixir, import: Kernel],
 [{:f, [context: Elixir], [{:x, [], Elixir}]}, [do: {:x, [], Elixir}]]}

The parser seems to use different rules in each case: in the def case, the parser just expands function calls into AST nodes according to what you call the “syntax” of the language. In the <<>> case, the parser uses some rules (different from just expanding function calls) and turns it into: {:<<>>, [], [{:a, [], Elixir}, {:b, [], Elixir}, {:c, [], Elixir}]}.

Those rules are the syntax.

Of course the parser doesn’t know that <<>> is a binary, it’s not it’s job. Iterpreting the AST generated by the parser is semantic analysis or something like that. The parser doesn’t even know that the :<<>> atom is non rebindable. But it does know that when it finds <<...>> it must output the AST [{:<<>>, [], [...]}].

48

It’s very cool! Just tried to quote some <<>> expression and the minus operators expand into actual minus operators inside the AST. I still contend that <<>> looks a lot like syntax (see above)

49

Sigils are very different from containers such as tuples, lists and binaries.

Yes. But the parser does not add meaning to this. You can also write this:

iex(1)> quote do: <<x :: def(foo, do: 1)>>
{:<<>>, [],
 [{:::, [],
   [{:x, [], Elixir},
    {:def, [context: Elixir, import: Kernel],
     [{:foo, [context: Elixir], Elixir}, [do: 1]]}]}]}

Therefore, syntax wise, <<>> can mean anything. That’s why I said you won’t find the documentation for what <<1::byte>> means in the syntax reference. The syntax allows you to put whatever you want on the right side of ::, the meaning comes from elsewhere.

This is a big contrast to other languages, such as Erlang, where what is allowed inside <<>> is a directly known by the parser.

1 Like
50

I would say that that is a language developers way of looking at it. :slight_smile: For most users of Elixir whether parts of the “standard syntax” like def and if are built-in syntax or macros is completely irrelevant. How I write them and how do I use them are the important bits. do ... end vs do: ... is a little more complex but I think you can happily just describe what they look like and how they equate to make most users happy and to get them right.

Again, I think most users, even those who write big complex systems, will not define their own macros even though they will be using lots of things defined by macros. In one way it is similar to the fact that using OTP you can write large extremely concurrent and parallel systems without a spawn, send or receive in sight. You should know that things are being done concurrently but you don’t need to use the explicit concurrency primitives.

51

Ok, thanks! I agree with you, then. The types inside the <<>> are certainly not syntax, and they shouldn’t be highlighted any differently in a syntax highlighter.

52

I agree. What a binary looks like is very much syntax, the same as for all the other data types. The syntax describes what things look like. What they are parsed to is basically irrelevant to most users.

I think the important thing is to understand for whom you are writing a syntax description.

1 Like
53

I don’t think we agree. You say it looks like it is syntax. I say it is syntax. It’s a sequence of symbols that’s translated into an AST in a very unconventional way compared to the rest of the language (the same goes for maps, tuples, etc).

You’ve written this, which makes me think you think these cases are analogous. My position is that def and if look like syntax (they are turned into an AST node the way any function call is), while <<...>> is syntax (the parser does funny stuff with it to turn it into an AST node).

Or am I misunderstanding your position?

EDIT: The individual expressions separated by commas inside of a <<...>> are certainly not syntax (but they do look like it, enough to fool me before trying quote a binary quote do <<...>> end, like José showed above).

54

I wonder if there should be two reference guides: one for Developers who don’t want to write macros and one for Developers who want to write macros, maybe titled something like:

  1. Reference for Developers who don’t like LISP or don’t know what it is, which basically describes elixir like if it were python. After all, it has def, if, with, Enum.reduce_while (basically a python for loop that doesn’t leak outside the scope). defmacro does not exist and the |> operator is a special form that does magic with your functions to put the arguments in the right place. Modules are like classes that have no internal state and can’t be instantiated, unless they inherit (sorry, use! they use!) GenServers and you mess with something arcane like the process registry. Then you get gets and sets and mutable internal object state. Ok, I’ll stop now :smile:

  2. Reference for Developers who think that LISP is the best thing ever and that the characters that compose the program are mere distractions from the holy truth of the AST, which describes elixir as if it were LISP and in which after chapter 2 all programs are written directly by manipulating the AST using functions (do ... end? defmacro? who needs them? We write our AST directly!)

Despite the joking nature of these titles, and the parody-like description, I’m being serious, and I’d totally buy a book like number two if someone were to write it. And if @rvirding’s right, then reference #1 might be useful for many people too.

1 Like
55

Mmmmmmmm
 ^.^

I mean, uh, cough


1 Like
56

Yes, you are misunderstanding my position.

We do agree. I did not say that “it looks like syntax”, what I said was “what a binary looks like” “is very much syntax”. I could flip the order and say syntax is what they look like.

Again I did not say looks like syntax, but what it looks like is syntax. And I see def and if as syntax as they are part of the standard definition of the Elixir language. In the same as data types.

1 Like
57

Ok, we do agree on this one.

Here I’m not sure we agree. The definition of syntax I use (it’s been a long time since I’ve read a book on the subject) is the same as the one given on Wikipedia (Syntax (programming languages) - Wikipedia):

In computer science, the syntax of a computer language is the set of rules that defines the combinations of symbols that are considered to be a correctly structured document or fragment in that language.

While semantics is (from the same article):

The syntax of a language describes the form of a valid program, but does not provide any information about the meaning of the program or the results of executing that program. The meaning given to a combination of symbols is handled by semantics (either formal or hard-coded in a reference implementation). Not all syntactically correct programs are semantically correct.

From the point of view of the syntax, there is nothing special about if and def. The rules you use to decide whether they are part of a valid program are the same. They are both ordinary identifiers.

I admit my perspective might be too pedantic, strict and theoretical. After all, you’re the one who has implemented (real, used by may people) languages in practice, so I certainly respect your mental models regarding this topic!

But in this case, the difference these definitions draw between Syntax and Semantics is useful to me, while your nebulous notion of “it’s syntax if most programmers agree it is” is to vague for me personally.

58

Just an aside, if really should not be a special form or parsed with the syntax or anything of the sort in Elixir, it could easily be just a macro like:

defmacro if(cond_ast, bodies) do
  then_body = bodies[:do]
  else_body = bodies[:else]
  quote do
    cond do
      unquote(cond_ast) -> unquote(then_body)
      true -> unquote(else_body)
    end
  end
end

Or something of that sort. There’s no reason for it to be anything but a macro in Kernel, oh and look it is (although it does some weird stuff like special case the random atom nil and such
 *cough* still not a fan of nil being special at all), but yea it is not syntax, nor is def as it is just a macro as well.

The point of this is, since they are macro’s then the user could override them to make them do something else (like how I override def in my defguard library to allow you to define super-guards like is_struct (I still think my style (though complete) of defguard should be built-in to elixir) (I might be in a lisp’y mood
)), but because of that syntax coloring them special may not always make sense, like say in a DSEL.

What I really really hate being special-form’d are things like with and for, both of those could have been done in other more traditionally ‘macro’ ways without the comma-splosion, but since they are special-formed then syntax coloring on them can be specialized