Composing macros (using a macro inside another macro)

Removing the __MODULE__. from othermacro fixes the require issue, but exposes a bigger problem:

defmodule Test do
  defmacro somemacro(arg1, do: block) do
    quote do
      defmodule unquote(arg1) do 
        def unquote(arg1), do: 1
        unquote_splicing(elem(block, 2))
      end
    end
  end

  defmacro othermacro(arg2, arg1, do: block) do
    newblock = quote do
      def unquote(arg2), do: 2
    end

    somemacro arg1, do: newblock
  end
end

which fails with:

** (ArgumentError) expected a list with quoted expressions in unquote_splicing/1, got: nil
    (elixir 1.11.2) src/elixir_quote.erl:185: :elixir_quote.argument_error/1
    (elixir 1.11.2) src/elixir_quote.erl:166: :elixir_quote.list/2

This happens because when othermacro calls somemacro, the value somemacro sees in block is {:newblock, [line: 17], nil}.

One way to avoid this problem is to keep the defmacro parts simple and depend on plain functions to generate the actual AST fragments:

defmodule Test do
  defmacro somemacro(arg1, do: block) do
    # wrapped block in a list because unquote_splicing expects a list of quoted expressions
    build_ast(arg1, [block])
  end

  defmacro othermacro(arg2, arg1, do: block) do
    newblock = quote do
      def unquote(arg2)(), do: 2
    end

    # the old code didn't use block (?) 
    build_ast(arg1, [block, newblock])
  end

  defp build_ast(arg1, block) do
    quote do
      defmodule unquote(arg1) do 
        def unquote(arg1)(), do: 1
        unquote_splicing(block)
      end
    end
  end
end

defmodule FooMod do
  require Test

  Test.othermacro(:foo, :bar, do: "not used")
end

BUT BEWARE SOME GOTCHAS

The module defined by Test.somemacro is not nested - so in the example, it generates a top-level module named :bar:

iex(26)> :"Elixir.FooMod.bar".foo
** (UndefinedFunctionError) function :"Elixir.FooMod.bar".foo/0 is undefined (module :"Elixir.FooMod.bar" is not available)
    :"Elixir.FooMod.bar".foo()

iex(26)> :bar.foo                
2

This may not be what you want.

right…

Why doesn’t this work?

defmodule Test do
  defmacro somemacro(arg1, do: block) do
    quote do
      defmodule unquote(arg1) do 
        def unquote(arg1), do: 1
        unquote(block)      
      end
    end
  end

  defmacro othermacro(arg2, arg1, do: block) do
    somemacro arg1, do: quote do
      def unquote(arg2), do: 2
      unquote(block)
    end
  end
end

Is it because the unquote is evaluated after being passed into somemacro?

It comes down to the difference between def and defmacro:

• def defines a function that expects evaluated arguments and returns a value

• defmacro defines a function that expects AST arguments and returns a value (that’s usually AST) that is then evaluated

Quick demo:

defmodule MacroTest do
  def plain_function(arg) do
    IO.inspect(arg, label: "plain function")
  end

  defmacro macro_function(arg) do
    IO.inspect(arg, label: "macro function")
  end
end

arg2 = :blam # will fail otherwise
MacroTest.plain_function(do: quote do
  def unquote(arg2), do: 2
end)

MacroTest.macro_function(do: quote do
  def unquote(arg2), do: 2
end)

The “plain function” version returns this AST:

prints: "[do: {:def, [context: Elixir, import: Kernel], [:blam, [do: 2]]}]"
returns: [do: {:def, [context: Elixir, import: Kernel], [:blam, [do: 2]]}]

the “macro function” version returns this AST:

prints: [
  do: {:quote, [line: 38],
   [
     [
       do: {:def, [line: 39],
        [{:unquote, [line: 39], [{:arg2, [line: 39], nil}]}, [do: 2]]}
     ]
   ]}
]
returns: [do: {:def, [context: Elixir, import: Kernel], [:blam, [do: 2]]}]

A key thing to observe: the argument to macro_function is an AST representing the arguments passed. For instance, extracting code to a variable does NOT work quite like you’d expect:

quoted_block = quote do   
  def unquote(arg2), do: 2
end

MacroTest.macro_function(do: quoted_block)
prints: [do: {:quoted_block, [line: 45], nil}]
returns: [do: {:def, [context: Elixir, import: Kernel], [:blam, [do: 2]]}]

Here macro_function receives an AST representing “access the local variable named quoted_block”.

Right this makes sense it’s about what’s there at compile time.
I guess when othermacro is compiled somemacro is already expanded so there’s no opportunity to evaluate the unquote.
In that case though, shouldn’t this work?

defmodule Test do
  defmacro somemacro(arg1, do: block) do
    quote do
      defmodule unquote(arg1) do 
        def unquote(arg1), do: 1
        unquote(block)
      end
    end
  end

  defmacro othermacro(arg2, arg1, do: block) do
    somemacro arg1 do
      def unquote(arg2), do: 2
      unquote(block)
    end
  end
end

That should expand to an AST that has further unquote definitions in it which are evaluated in the context of othermacro, right? (this isn’t right, it throws an error). I guess what I want to do is quote an unquote operation, so that the expanded macro has an unquote operation in it? Not sure how to do that.

I don’t understand how using functions helps - it’s the same issue where I want to compose ASTs. So I want, ideally, to have a function that returns an AST and then another function that adds some extra stuff to that AST. because there is no way to do that directly (is there some AST function I don’t know about), I’m left with composing macros.

Is this what Macro.escape is for? I can define the value of the block and escape it into the macro.

defmodule Test do
  defmacro somemacro(arg1, do: block) do
    quote do
      defmodule unquote(arg1) do 
        def unquote(arg1), do: 1
        unquote(block)
      end
    end
  end

  defmacro othermacro(arg2, arg1, do: block) do
    newblock = quote do
      def unquote(arg2), do: 2
      unquote(block)
    end
    somemacro arg1, do: Macro.escape(newblock)
  end
end

I end up with an invalid quoted expression though

** (CompileError) iex: invalid quoted expression: {:module, :one, <<70, 79, 82, 49, 0, 0, 4, 176, 66, 69, 65, 77, 65, 116, 85, 56, 0, 0, 0, 127, 0, 0, 0, 14, 3, 111, 110, 101, 8, 95, 95, 105, 110, 102, 111, 95, 95, 10, 97, 116, 116, 114, 105, 98, 117, 116, 101, ...>>, {:{}, [], [:__block__, [], [{:{}, [], [:def, [context: Test, import: Kernel], [:two, [do: 2]]]}, {:{}, [], [:__block__, [], []]}]]}}

Please make sure your quoted expressions are made of valid AST nodes. If you would like to introduce a value into the AST, such as a four-element tuple or a map, make sure to call Macro.escape/1 before
    expanding macro: Test.othermacro/3
    iex:31: (file)