Wait, is this correct about macros - when writing Elixir, we are writing code and values that get interpolated right into the AST?

I just read the “Metaprogramming” book by Chris McCord and I read something that messed me up. He says, “Macros are code that writes code. Their purpose in life is to interact with the AST using Elixir’s high-level syntax.”

Am I crazy or did I just realize that when writing Elixir, we are really just writing code and values that get interpolated right into the AST?

defmodule, def, defp…all of it! Direct interpolation of our code into the underlying AST?

3 Likes

The AST, Abstract Syntax Tree is a structured representation of the syntax of the language. So I guess :smiley:

I am not sharing the blown mind but it may be that I had the same reaction at some point :slight_smile:

1 Like

Yes, more or less that’s it. Hence, many people called Elixir “LISP-y”.

1 Like

Just to mind-blow a bit further: you can write macro’s that write macro’s too! How lovely :slight_smile:

Read tip to further explore and learn macros: The Erlangelist - Understanding macros, part 1

5 Likes

To blow the mind even further, you can also manipulate existing values that were defined in the original codebase, I would really like to see this be abused by libraries :joy:

2 Likes

Macros are, at the core, far simpler than folks realize. Macros themselves are not complex, but sometimes the semantics of quote and unquote can be, and I think thats often a point of confusion for folks in terms of how macros work.

Consider this macro

defmodule Mod do
  defmacro add(a, b) do
    quote do
      unquote(a) + unquote(b)
    end
  end
end

If you haven’t seen macros like this before, it can look pretty magical. It’s not clear that this is an AST template, and that unquote is a way to interpolate values into that template.

But if you peel back the covers a bit, it actually gets much clearer. There is no rule that a macro must use quote

defmodule Mod do
  defmacro add(a, b) do
    {:+, [], [a, b]}
  end
end

If you put those both in a project you will see that they do the same thing. There are subtle differences not worth going into that make it advantageous to use quote in general. But when you understand that a macro is just a function that is called at compile time that returns AST, most of the magic goes away.

Something very useful for understanding how elixir AST works in general is to just open up iex and return the result of quoteing some code for example:

iex(1)> quote do: a + b
{:+, [context: Elixir, imports: [{1, Kernel}, {2, Kernel}]],
 [
   {:a, [], Elixir},
   {:b, [context: Elixir, imports: [{1, IEx.Helpers}]], Elixir}
 ]}
11 Likes

I hear you. I was just blown away to realize how Elixir was working in comparison to other languages.

Reading it now…thx!

Ok, so Elixir offers a syntax that abstracts away a LISP-y nature.

2 Likes

Very interesting! Just when I thought I knew what was going on, I find out that there’s even more right under the hood.

Elixir is definitely a LISP in a mask :laughing:

4 Likes

My lib Routex (still alpha) rewrites the Phoenix Routes that are saved in attributes during compilation and replaces them. Does that count?

The extensions extensions:

  • make an alternative variant of sigil ~p which uses the original one (a macro calling a macro creating code)
  • create pattern matches which are then used for function heads in which the body uses the values from the pattern match in another pattern.
  • inspect the AST to define which variable to use (assigns, socket or conn) based on what is available in the caller module

There is not much you can’t do with AST/Macro’s.

3 Likes

This is indeed what makes it challenging. Especially when nesting comes into play. It helps (for starters) to write the code whey would like to ‘see’ in a module, then ‘quote do’ it and place ‘unquote’ around any dynamic parts (not to way ‘variable’).

Then you learn that you sometimes need Macro.escape, play with ‘keep_lines’, fail with MODULE a few times and well….that’s about it :slight_smile:

5 Likes

Yeah that’s exactly it. Understanding that macros are basically code generators but you don’t get to see the source code (unless you jump through extra hoops) is what makes it click for many people. But I really wish quote and unquote were much more obvious in how they should or should not be used i.e. more compiler warnings or even a built-in tool that vets macros that can tell you “you are using unquote in two-layer deep quote chain, check your hierarchy because it will likely not do what you expect”. Or special symbols, or special highlighting in editors etc.

The problem with macros is usually that you can’t easily understand what exactly did you screw up. I don’t remember more than 2 things I was truly unable to do with macros for 8.5 years with Elixir but I do very clearly remember 4-hours long hair-pulling sessions and at the end of one I wanted to gouge my eyes out after shouting “frak, I should have used Macro.escape on this quoted variable / expression here!”.

1 Like

Ok, so I just had my Elixir matrix moment with realizing that nearly everything is a macro and all of it was masking a LISPy nature. I just came across this video, and that helped me to put this all into perspective.

The Upside Down Dimension of Elixir - An Introduction to Metaprogramming

Thanks peeps.
Much respect.

1 Like

Macros are one of those things that feel really confusing at first, but once you understand them they feel simple and you don’t remember why it was so confusing.

The thing that helped me understand unquote, for example, was seeing the analogy with string interpolation: just like you can use #{} to interpolate a value into a string "like #{this}", you can use unquote to interpolate a value into a quote:

quote do
  "like" <> unquote(this)
end

So you can’t use unquote outside of quote just like it doesn’t make sense to use #{} outside of a string.

I wrote a series of posts trying to explain Elixir macros in my own words, I hope someone here finds it helpful:

5 Likes

Sweet. Great write up!

1 Like