How to access module attributes in a macro outside `quote`?

Questions / Help
,
1

In a macro, can you access a module attribute outside quote?. I know about Module.get_attributes/2 but I can only get it to work inside quote. Outside quote, it always returns nil whenever I have tried.

Here’s an example:

defmodule MyMacros do
  defmacro my_macro() do
    module = __CALLER__.module
    Module.get_attribute(module, :foo) |> IO.inspect(label: "outside quote")

    quote do
      Module.get_attribute(unquote(module), :foo) |> IO.inspect(label: "with unquoted module")
      Module.get_attribute(__MODULE__, :foo) |> IO.inspect(label: "with __MODULE__")
    end
  end
end

defmodule UseMacros do
  require MyMacros

  @foo bar: 1
  MyMacros.my_macro()
end

(On a side note: I formatted this with normal indentation, but the forum seems to render it incorrectly for some reason…)

Here’s the output this produces:

outside quote: nil
with unquoted module: [bar: 1]
with __MODULE__: [bar: 1]

This demonstrates that __CALLER__.module is correctly getting the module, but when I try to use it outside quote to get the module attribute, I get back nil.

Any ideas why or if there’s any way to get this to work?

1 Like
2

The attributes do not really ‘exist’ at this time yet unless they are registered, thus what you could do is change require to use at the call site, and make a __using__ macro in your module that register_attribute/3's the names that are needed, then in your my_macro/0 you can use Module.get_attribute(__CALLER__.module, :blah_attribute_name) or so to get the value, I think. :slight_smile:

I think that put_attribute/3 might be required though, but you can wrap that up in another macro too if needed.

1 Like
3

The attributes do not really ‘exist’ at this time yet unless they are registered

This does not make sense to me. The docs about register_attribute/3 say nothing about making the attributes exist; it just provides a way to set the accumulate and persist options from my reading. If you’re not wanting to change those options for a module attribute, it doesn’t seem like Module.register_attribute/3 does anything. Also, why does quote make the module attribute “exist” when it did not previously? I thought the module attribute existed as soon as it was declared.

Anyhow, I updated my example to use Module.register_atttribute/3:

defmodule MyMacros do
  defmacro __using__(_) do
    quote do
      Module.register_attribute(__MODULE__, :foo, persist: true)
    end
  end

  defmacro my_macro() do
    module = __CALLER__.module
    Module.get_attribute(module, :foo) |> IO.inspect(label: "outside quote")

    quote do
      Module.get_attribute(unquote(module), :foo) |> IO.inspect(label: "with unquoted module")
      Module.get_attribute(__MODULE__, :foo) |> IO.inspect(label: "with __MODULE__")
    end
  end
end

defmodule UseMacros do
  use MyMacros

  @foo bar: 1
  MyMacros.my_macro()
end

…and it produces the exact same result.

I think that put_attribute/3 might be required though, but you can wrap that up in another macro too if needed.

I tried that and I get the same result.

4

It doesn’t, rather putting it in a quote moves the actual attribute lookup from compile-time to run-time. :slight_smile:

Yep, instead of @foo bar: 1 you need Module.put_attribute(__MODULE__, :bar, 1), or wrap that up in a macro too to make it shorter and easier. :slight_smile:

Really? It works here:

defmodule MyMacros do
  defmacro __using__(_) do
    quote do
      Module.register_attribute(__MODULE__, :foo, persist: true)
    end
  end

  defmacro blah([{key, value}]) do
    module = __CALLER__.module
    Module.put_attribute(module, key, value) |> IO.inspect(label: "setting attribute")
    Module.get_attribute(module, key) |> IO.inspect(label: "getting attribute")
    quote do # Also put it back in so it is accessible at run-time
      @foo unquote({key, value})
    end
  end

  defmacro my_macro() do
    module = __CALLER__.module
    Module.get_attribute(module, :foo) |> IO.inspect(label: "outside quote")

    quote do
      Module.get_attribute(unquote(module), :foo) |> IO.inspect(label: "with unquoted module")
      Module.get_attribute(__MODULE__, :foo) |> IO.inspect(label: "with __MODULE__")
      @foo |> IO.inspect(label: "plain")
    end
  end
end

defmodule UseMacros do
  use MyMacros

  MyMacros.blah foo: [bar: 1]
  MyMacros.my_macro()
end

Results in:

setting attribute: :ok
getting attribute: [bar: 1]
outside quote: [bar: 1]
with unquoted module: {:foo, [bar: 1]}
with __MODULE__: {:foo, [bar: 1]}
plain: {:foo, [bar: 1]}
5

It doesn’t, rather putting it in a quote moves the actual attribute lookup from compile-time to run-time. :slight_smile:

That’s a useful distinction, but I don’t understand why code outside the quote is evaluated at compile-time while code inside it is evaluated at runtime.

For example, consider this module:

defmodule MyModule do
  k = 1 + 2
  IO.puts("In #{__MODULE__}: #{k}")
end

According to my understanding, this code is evaluated at compile time, not run time. In fact, if I run mix compile on it, it prints MyModule: 3 confirming that it is evaluated as part of compilation (which is what “compile time” is, right?). If I move that code into a macro outside a quote, it produces the same result:

defmodule MyMacros do
  defmacro print_stuff do
    module = __CALLER__.module
    k = 1 + 2
    IO.puts("In #{module}: #{k}")
  end
end

defmodule MyModule do
  require MyMacros
  MyMacros.print_stuff
end

Again, when I run mix compile, it prints MyModule: 3, confirming the code is evaluating at compile time.

So in what sense is code in a macro outside of a quote not evaluated at compile time, given it evaluates as part of compilation?

Really? It works here:

That’s interesting your example worked. Here’s what I had tried:

(For some reason, I can’t get this to format how I want: I put the quote and my reply in separate paragraphs but the forum is collapsing it into the prior paragraph. Any idea why?)

defmodule MyMacros do
  defmacro __using__(_) do
    quote do
      Module.register_attribute(__MODULE__, :foo, persist: true)
    end
  end

  defmacro my_macro() do
    module = __CALLER__.module
    Module.get_attribute(module, :foo) |> IO.inspect(label: "outside quote")

    quote do
      Module.get_attribute(unquote(module), :foo) |> IO.inspect(label: "with unquoted module")
      Module.get_attribute(__MODULE__, :foo) |> IO.inspect(label: "with __MODULE__")
    end
  end
end

defmodule UseMacros do
  use MyMacros

  Module.put_attribute(__MODULE__, :foo, bar: 1)
  MyMacros.my_macro()
end

Basically, just replacing @foo bar: 1 with Module.put_attribute(__MODULE__, :foo, bar: 1).

I need to play around with your some more to see why it works while mine doesn’t.

6

Because code inside a quote is not run, the AST of the code is returned instead:

iex> quote do
...> blah bleh dorp
...> end
{:blah, [], [{:bleh, [], [{:dorp, [], Elixir}]}]}

So when your module returns the quote’s return, it is just returning AST, which is then just put in place in the module at the location of the module call, it is not executed until runtime as it is just code at this point in AST form. :slight_smile:

Teeeechnically the IO.puts/1 there is put into the module’s AST, it gets executed at ‘run-time’, but the run time in this case is the module definition call, the defmodule/2 call at the top is actually a macro (in essence) that takes the body of the do as an AST and passes it to the elixir module compilation function, that function runs over the do body of the module definition and executes each expression in turn, so IO.puts/1 will print at this point, a def will call the function definition function that actually creates the function to be run (passing that function the def’s do block as an AST), etc… :slight_smile:

Remember that mix compile has a runtime of its own, and that is where the module and function and such definitions are actually executed. It is a bit more defined then how I represented it above, but that is the gist of it.

Discourse is standard markdown, so use code-fences:
```elixir
Properly formatted elixir code here
```
Turns in to:

Properly formatted elixir code here

But it did not work because your Module.put_attribute(__MODULE__, :foo, bar: 1) line is being set as a call into the already compiled module AST rather than being run before your macro is run because put_attribute/3 is a def, not a defmacro, if it were a defmacro then it would run as you expect:

:slight_smile:

Macro’s are a black art, no matter the language you use, they require that you understand not only the language well, but also ‘how’ it is compiled in pretty good detail. ^.^

2 Likes
7

I’m aware of that. Conceptually, code in a quote in a macro gets injected into the call site, as I understand it. But code in a module definition is evaluated as part of compilation, it’s not clear to me why code injected into a module definition from a macro quote is evaluated at “runtime” when it is clearly evaluated as part of the compilation process.

Remember that mix compile has a runtime of its own, and that is where the module and function and such definitions are actually executed.

If that’s true, then how does this work?

defmodule Foo do
  Enum.each [foo: 1, bar: 2], fn {k, v} ->
    def unquote(k)(), do: unquote(v)
  end
end

This dynamically defines a Foo.foo/0 function and a Foo.bar/0 function. From what you’ve said, the Enum.each/2 is not evaluated at compile time, running instead at runtime. But within the enumeration we are defining functions. So apparently functions are being defined and compiled at runtime instead of compile time? If module function compilation happens outside of “compile time” then what does “compile time” as a concept even mean?

Does the distinction you’re making have to do with macro expansion phases? That makes more sense to me then compile time vs runtime (given you can use “runtime” code to define functions for compilation, the distinction doesn’t make sense to me at all), and it’s something I’ve seen referenced in some Elixir books and documentation.

Discourse is standard markdown, so use code-fences:

Yep. I’ve been doing that from the start (as it’s what I’m used to from years of doing it on GitHub). Still not rendering correctly for me, though :(.

But it did not work because your Module.put_attribute(__MODULE__, :foo, bar: 1) line is being set as a call into the already compiled module AST rather than being run before your macro is run because put_attribute/3 is a def, not a defmacro, if it were a defmacro then it would run as you expect:

So, translating this into the concept of macro expansion phases, it sounds like my macro is being expanded during an expansion phase that happens before whatever @attribute value expands into is evaluated (or before put_attribute/3 is evaluated, if I call that directly). Is that accurate?

@josevalim I think there’s a bit of a whole in the documentation regarding this stuff. I’ve read @chrismccord’s Metaprogramming Elixir and the Elixir docs regarding macros and quote and I don’t really think it provides the necessary concepts to understand this kind of issue (at least not for how my brain thinks, apparently). Are there some simple ways we can improve the docs regarding this? I’m thinking maybe a doc explaining macro expansion phases in more detail and how evaluation order may not match what you’d expect. And also maybe the docs on the Module attribute functions could mention this gotcha. It definitely surprised me.

1 Like
8

Because that code itself is not executed when the AST is generated, rather it is executed after the AST is generated (the Enum.each call and all are in that AST, not run yet), then passed to the defmodule internal call that generates the module, it goes over the AST and anything it does not understand (special forms like the internal ‘def’ type) it executes and takes the result of like a macro, this is the step at which a ‘function’ (not a macro) call gets executed during module definition. The Enum.each returns the new ast (since the def part became ast at that point in a way) and the module compiler then uses ‘that’. :slight_smile:

Yes it is convoluted, macros often are. ^.^

Its not compiled at the ‘mix’ compile time correct, it is compiled at the ‘mix’ runtime, which is the time that mix runs all its code to do things like compiling a module to a beam file. :slight_smile:

If you want to use program runtime (not mix runtime) code to generate a module you have to use the compile function and pass the AST itself to it. Or eval text too. ^.^

Huh, if you have a reproducable example you should send it to the meta discourse forum (the discourse forum that is hosted in discourse on the discourse website), they hop on bugs like that! ^.^

Basically, since the put_attribute was in the module area but put_attribute is a function, not a module, it got executed when the module got compiled, not when the module AST was generated, so it was too late for your macros (that are executed at ast generation (in essence, there are things about this too)) to see the result of its call.

I have no doubt they would love good documentation on this PR’s into Elixir! Macro’s are a hard hard topic, I figured out most of Elixir’s by reading the erlang source of Elixir itself, and even then there are so many different cases it is still hard to hold it all in my head (as is common in macro systems). ^.^

9

This may be the source of confusion. Code in Elixir is not evaluated (executed line by line) except on IEx. It is always compiled and then executed. This means that, if you have three lines:

foo_macro()
bar_macro()
baz_macro()

First we expand foo_macro(), then we expand bar_macro(), and then baz_macro() and just then we execute their contents. This means that @attr :bar won’t be seen inside any macro because it has not been executed yet.

5 Likes