Macro expansion order

Module attributes are macros as well: Kernel — Elixir v1.20.2 and It seems one case does expand to what the @/1 macro expands to in terms of AST, while the other seems to expand to the result value. The difference might come from the difference between how defmodule handles it’s do: block vs. what the def macro does with it.

Can you elaborate on the difference between handling do: block in both? Is it documented somewhere or is it just a wild guess like mine?

There are differences like this one, but I’m not sure if they are really related to what you’re seeing:

I believe the information you are missing is that def is a macro that stores some AST to be expanded and evaluated. So when a function body is expanded, the module body has been expanded and executed.

You can imagine your module, after expansion, becomes this:

defmodule Main do
  :elixir_internal.store_module_attribute(:my_attr, :asdf)
  MyMacro.print_module_attribute(:elixir_internal.read_module_attribute(:my_attr))
  :elixir_internal.store_function(:def, :in_function, quote do
    MyMacro.print_module_attribute(@my_attr)
  end)
end

Generally speaking, we should avoid doing work inside expansion.

And the reason why we delay expanding function bodies is because we don’t want to expand them if they are not meant to be compiled. Imagine you wrap your functions in an if block that does not evaluate to true. By postponing it, we avoid doing unnecessary work. The late expansion is also what allows dynamic definitions.

In another post, Jose Valim stated that all macros are expanded in order–before any executable code contained in a macro is executed. After all the macros have been expanded, then any executable code in each macro is executed. As a result, inside the macro expansions that follow the macro @my_attr :whatever, the value :whatever will not have been set yet (by the executable code contained in the @ macro), so the subsequent macro expansions use the value nil.

So when a function body is expanded, the module body has [already] been expanded and executed .

So the def macro is expanded at compile time, but the macros inside a def are not? Are you saying that the macros found inside a def are not expanded until the function is actually called at runtime?

In Metaprogramming Elixir on p. 13 it says:

When the compiler encounters a macro, it recursively expands it until the code no longer contains any macro calls.

The whole reason we programmers buy metaprogramming books is to learn the details not provided in the guides. In Metaprogramming Elixir, there is no mention of the delayed evaluation of macros inside a def.

def macros need to expand at compile time. The compilation has a couple of phases and module attributes are available only after the compilation finished. If I understood Jose’s answer correctly, it works like this:

1. Expand all macros in order recursively. In my example, it works like this:

• require MyMacro makes sure that MyMacro module is already fully compiled
• @my_attr: :asdf becomes “hey! register this attribute when evaluating final AST”
• then expands the the macro MyMacro.print_module_attribute(@my_attr); at current point, attributes are not calculated (it prints “ATTRIBUTE: …” and "EXPANDED VALUE: "
• then expands def macro but it is kind of special. It uses Macro.escape under the hood so it is not yet fully expanded. Again, it is something like register that AST under that function name

2. Evaluate the module (it is not runtime!). Now we have the body so we can start evaluating:

• calculate module attributes
• evaluate first expanded macro (it evaluated to nil so nothing happens)
• evaluate function definitions; Since they had the expansion delayed, we need to expand them now and that prints the second “ATTRIBUTE: …” and “EXPANDED VALUE: …”. Attributes were already evaluated.

I hope I get it right now

I’d love to see this mechanism explained in greater detail in “Metaprogramming Elixir”!

Yes! And another thing the book omits: ALL the differences between bind_quoted() and unquote(), which is something that really tripped me up. For instance, bind_quoted() does NOT make the specified variables available inside a def in the quote block, where unquote() does. I don’t think I’ll ever use bind_quoted() because of that feature. The book makes it seem like bind_quoted() is a drop in replacement for unquote(), whereas, as far as I can tell, bind_quoted() is merely equivalent to:

defmacro two(x) do
  quote do
    binded_x = unquote(x)  #You get one evaluation of x like with bind_quoted().

    #Now use binded_x where it can be seen, which is not inside a def, like calc():
    IO.puts "The macro arg in two() was: #{binded_x}."

    def calc do
      IO.puts "The macro arg in two() was:  Can't see it!"
    end

  end
end

I see no reason to use bind_quoted() when it can cause so much pain when used as a drop in replacement for unquote(). I’ll just use the longcut above, so that I’ll know what’s going on.

The problem here is not that bind_quoted is not working, but that unquote fragments within macros quote do need „two levels“ of unquoting (outside quote → inside quote and inside quote → inside function). That‘s exactly the place you even need bind_quoted to resolve the ambiguity of which task unquote is fulfilling.

I believe @sasajuric talks about this in his series but it has been a while since I last read it: The Erlangelist - Understanding macros, part 1

bind_quoted actually disables unquoting for the current quote, this is mentioned in the docs. There are actual use cases for it, such as handling two level of unquotes, as mentioned by @LostKobrakai.

I’m not sure what you are trying to say there, but I don’t need two levels of unquoting here:

defmodule My do

  defmacro one(x) do
    quote do
      def show_in_one do
        IO.puts "The macro arg for one() was: #{unquote(x)}"
      end
    end

  end
end