Hey @kip , so sorry for the delayed response, I was away for a couple of days.
Thank you so much for your answer! It actually helped me to solve my conundrum. After you emphasized how I should think about macros as regular functions, everything clicked.
This was the final explanation I had come up with (bits are extracted from the article I am working on):
The compilation process can be broken down into:
Parsing phase - Elixir source code (program) is parsed into an AST, which we will call the initial AST
Expansion phase - initial AST is scanned and macro calls are identified
Macros are executed so that their output (AST) can be injected into and expanded at the callsite
Expansion occurs recursively and a final AST is generated
Bytecode generation phase - after the final AST has been generated, the compiler performs an additional set of operations that eventually generates BEAM VM bytecode which is then executed
Macros contain two contexts: a macro context and a caller context.
defmacro foo do
# This is the macro's context, this is executed when the macro is called
# This is the return value of the macro (AST)
# This is the caller's context, this is executed when the callsite is called
As you can see, the macro’s context is any expression declared before the
quote. The caller’s context is the behavior that is declared in the
quote; the AST generated from
quote is the output of the macro and is the AST that is injected into and expanded at the callsite, hence why it is referred to as the caller’s context.
First, let’s review what we understand about macros. They are a compile-time construct that receives ASTs as input and returns ASTs as output. Aside from these two attributes, they behave just like any other function.
Knowing this, we can move on to the first component of the expansion phase, “Macros are executed so that their output (AST) can be injected into the callsite and expanded at the callsite”. In order for the compiler to know what AST needs to be injected into the callsite, it has to retrieve the output of the macro (since it is an AST). In order to retrieve the output of the macro, it has to be executed - just like any other function; and this process has to be done during compilation as that is when the expansion phase occurs. The macro call will be parsed as an AST during the parsing phase and the compiler identifies and executes these macro call ASTs during compilation, prior to the expansion phase.
If we think of macros as regular functions, the macro context will be the function body and the caller context will be the result of the function. Understanding this, the behavior exhibited above makes sense. During compilation, a macro has to be executed so that its result can be retrieved. This causes the macro context to be evaluated during compilation. Once the macro has been evaluated, the caller context is injected into and expanded at the callsite of the macro. This explains why the macro context and caller context have different “owners”. As the macro context is evaluated during compile-time and treated as a regular function body, it is executed within its containing module. Thus, it is “owned” by its containing module. The caller context is injected into the callsite and evaluated whenever the callsite is evaluated, thus, it is “owned” by the caller.
TL;DR Macro context is evaluated before the formal expansion phase of the compiler. This is so that the output AST of the macro can be injected into and expanded at the callsite during the actual expansion phase
Once again, thank you so much!