Quote()'ed expressions and functions

Questions / Help
1

I’m reading the Elixir tutorial on metaprogramming, and in the section on quote() and unquote() it says:

…it is important to make a distinction between a regular Elixir value (like a list, a map, a process, a reference, etc) and a quoted expression. Some values, such as integers, atoms, and strings, have a quoted expression equal to the value itself. Other values, like maps, need to be explicitly converted. Finally, values like functions and references cannot be converted to a quoted expression at all.

Then what’s this:

iex(8)> result = quote do: fn(x) -> x+2 end        
{:fn, [],
 [
   {:->, [],
    [
      [{:x, [], Elixir}],
      {:+, [context: Elixir, import: Kernel], [{:x, [], Elixir}, 2]}
    ]}
 ]}

iex(9)> Macro.to_string(result)            
"fn x -> x + 2 end"
2

That’s an AST representation of a function, what it means is something like unquoting a function into an ast is invalid. Do note that elixir’s AST can represent it, and it will be interpreted as such, but on compilation it will fail.

3

Thats only the AST of an anonymous function, not the function itself.

The next example does show it a bit better, but not perfectly, at least to my understanding.

iex(1)> fun = fn -> 0 end
#Function<20.127694169/0 in :erl_eval.expr/5>
iex(2)> quote do: unquote(fun)
#Function<20.127694169/0 in :erl_eval.expr/5>
iex(3)> Macro.to_string(v(2))
"#Function<20.127694169/0 in :erl_eval.expr/5>"
4

Compilation doesn’t fail here:

defmodule My do
  defmacro go() do
    quote do
      def hello(), do: IO.puts "hello"
    end
  end
end

defmodule Test do
  require My
  My.go()
end

Test.hello

Output:

~/elixir_programs$ iex macros2.ex

Erlang/OTP 20 [erts-9.3] [source] [64-bit] [smp:4:4] [ds:4:4:10] [async-threads:10] [hipe] [kernel-poll:false]

hello

Interactive Elixir (1.6.6) - press Ctrl+C to exit (type h() ENTER for help)

iex(1)>

Yet, I quoted a function.

5

No, you quoted the definition of a function, not a function itself…

6

This.

Quoting a function would be like:

iex(6)> defmodule Blah do
...(6)>   defmacro boop do
...(6)>     quote do
...(6)>       unquote(fn x -> x end)
...(6)>     end
...(6)>   end
...(6)> end
{:module, Blah,
 <<70, 79, 82, 49, 0, 0, 4, 40, 66, 69, 65, 77, 65, 116, 85, 56, 0, 0, 0, 150,
   0, 0, 0, 14, 11, 69, 108, 105, 120, 105, 114, 46, 66, 108, 97, 104, 8, 95,
   95, 105, 110, 102, 111, 95, 95, 7, 99, ...>>, {:boop, 0}}
iex(7)> require Blah
Blah
iex(8)> Blah.boop()
** (CompileError) iex: invalid quoted expression: #Function<0.57066798/1 in Blah."MACRO-boop"/1>
    expanding macro: Blah.boop/0
    iex:8: (file)

I’m pretty sure it would fail serialization to the beam files as well.

EDIT: In fact it does:

iex(8)> defmodule Bleep do
...(8)>   def boop, do: unquote(fn x -> x end)
...(8)> end
** (CompileError) iex: invalid quoted expression: #Function<0.3153080 in file:iex>
    iex:9: (module)
7

So something like this?

defmodule My do
  defmacro go(fun) do
    quote do
      def hello(), do: unquote(Macro.escape(fun) )()
    end
  end
end

defmodule Test do
  require My
  My.go(fn() -> IO.puts "hello" end)
end

Test.hello

Output:

~/elixir_programs$ iex macros2.ex 
Erlang/OTP 20 [erts-9.3] [source] [64-bit] [smp:4:4] [ds:4:4:10] [async-threads:10] [hipe] [kernel-poll:false]

** (CompileError) macros2.ex:11: invalid call {:fn, [line: 11], [{:->, [line: 11], [[], {{:., [line: 11], [{:__aliases__, [line: 11], [:IO]}, :puts]}, [line: 11], ["hello"]}]}]}()
    macros2.ex:11: (module)
8

I wouldn’t expect that to work because it’s my understanding that unquote() injects the very thing you give it into the ast–in particular, unquote() does not convert the thing you give it into an ast before injecting. In the macro examples that I’ve looked at, it can be hard to discern that unquote() doesn’t convert its argument to an ast because those examples often unquote something, like :hello or 10, where their ast’s are exactly the same as their values, so injecting the value is identical to injecting the ast of the value. Or, very importantly, in some macro examples unquote() is used on the arg to the macro–but args passed to a macro are converted to an ast automatically!

Here’s an example that may make you think that unquote() does convert to ast:

defmodule My do
  defmacro go(map) do
    quote do
      def hello(), do: unquote(map)
    end
    |> IO.inspect
  end
end

defmodule Test do
  require My
  My.go(%{a: 1, b: 2})
end

Test.hello

Output:

~/elixir_programs$ iex macros2.ex
Erlang/OTP 20 [erts-9.3] [source] [64-bit] [smp:4:4] [ds:4:4:10] [async-threads:10] [hipe] [kernel-poll:false]

{:def, [context: My, import: Kernel],

[{:hello, [context: My], []}, [do: {:%{}, [line: 12], [a: 1, b: 2]}]]}

Interactive Elixir (1.6.6) - press Ctrl+C to exit (type h() ENTER for help)
iex(1)>

You can see the proper ast for a map in there:

{:%{}, [line: 12], [a: 1, b: 2]}

But, an argument to a defmacro function, like the map, is automatically converted to an ast, so unquote() is inserting the map ast into the program ast, and all is well. Compare to:

defmodule My do
  defmacro go() do
    map = %{a: 1, b: 2}
    quote do
      def hello(), do: unquote(map)
    end
    |> IO.inspect
  end
end

defmodule Test do
  require My
  My.go()
end

Test.hello

Output:

~/elixir_programs$ iex macros2.ex 
Erlang/OTP 20 [erts-9.3] [source] [64-bit] [smp:4:4] [ds:4:4:10] [async-threads:10] [hipe] [kernel-poll:false]

{:def, [context: My, import: Kernel],
 [{:hello, [context: My], []}, [do: %{a: 1, b: 2}]]}

** (CompileError) macros2.ex: invalid quoted expression: %{a: 1, b: 2}
    macros2.ex:13: (module)
iex(1)>

In this case, unquote() inserted non-ast code into the program ast, which corrupted the program ast.

In your example, because fn x -> x end isn’t an ast, I would expect when you inject that code into an ast you would end up corrupting the ast.

9

That’s exactly what the documented paragraph means, you can’t encode a function or reference or pid or so in to the AST nor can you ‘escape’ them in to the AST in any way. :slight_smile:

10

This is perhaps best seen by trying to use Macro.escape:

iex(1)> Macro.escape(%{foo: 1})
{:%{}, [], [foo: 1]}
iex(2)> Macro.escape(fn -> 1 end)
** (ArgumentError) cannot escape #Function<20.127694169/0 in :erl_eval.expr/5>. The supported values are: lists, tuples, maps, atoms, numbers, bitstrings, PIDs and remote functions in the format &Mod.fun/arity
    (elixir) src/elixir_quote.erl:114: :elixir_quote.argument_error/1
    (elixir) src/elixir_quote.erl:136: :elixir_quote.escape/3
    (elixir) lib/macro.ex:422: Macro.escape/2
1 Like
11

Just like with OvermindDL1’s example, I wouldn’t expect that to work because you are using unquote() to inject non-ast code, namely fn -> 0 end, into an ast, which corrupts the ast.

12

Yeah, but as he said about this example, thats exactly what the docs are trying to explain.

1 Like
13

I don’t agree. I think you have to write something like the following to demonstrate what the docs are trying to say:

defmodule My do
  defmacro go(fun) do
    quote do
      def hello(), do: unquote(Macro.escape(fun) )()
    end
  end
end

defmodule Test do
  require My
  My.go(fn() -> IO.puts "hello" end)
end

Test.hello

I would expect that to work–because it injects ast into ast…but as the docs say, it won’t. Or, a more succinct example is benwilson512’s answer.

14

Correct, this code won’t:

iex(8)> defmodule My do
...(8)>   defmacro go(fun) do
...(8)>     quote do
...(8)>       def hello(), do: unquote(Macro.escape(fun) )()
...(8)>     end
...(8)>   end
...(8)> end
{:module, My,
 <<70, 79, 82, 49, 0, 0, 4, 188, 66, 69, 65, 77, 65, 116, 85, 56, 0, 0, 0, 152,
   0, 0, 0, 17, 9, 69, 108, 105, 120, 105, 114, 46, 77, 121, 8, 95, 95, 105,
   110, 102, 111, 95, 95, 7, 99, 111, 109, ...>>, {:go, 1}}
iex(9)> 
nil
iex(10)> defmodule Test do
...(10)>   require My
...(10)>   My.go(fn() -> IO.puts "hello" end)
...(10)> end
** (CompileError) iex:12: invalid call {:fn, [line: 12], [{:->, [line: 12], [[], {{:., [line: 12], [{:__aliases__, [line: 12], [:IO]}, :puts]}, [line: 12], ["hello"]}]}]}()
    iex:12: (module)

But not for the reason you think, it’s not that it is a function, because it’s not, it’s still only AST here, rather in go:
fun is:

{:fn, [],
 [
   {:->, [],
    [
      [],
      {{:., [], [{:__aliases__, [alias: false], [:IO]}, :puts]}, [], ["hello"]}
    ]}
 ]}

And thus Macro.escape(fun) returns:

{:{}, [],
 [
   :fn,
   [],
   [
     {:{}, [],
      [
        :->,
        [],
        [
          [],
          {:{}, [],
           [
             {:{}, [],
              [
                :.,
                [],
                [{:{}, [], [:__aliases__, [alias: false], [:IO]]}, :puts]
              ]},
             [],
             ["hello"]
           ]}
        ]
      ]}
   ]
 ]}

The AST was escaped into even more AST, so with the unquote that removes the outer AST layer into the ast, then that essentially becomes this code:

def hello(), do: {:fn, [],
 [
   {:->, [],
    [
      [],
      {{:., [], [{:__aliases__, [alias: false], [:IO]}, :puts]}, [], ["hello"]}
    ]}
 ]}()

Notice the () at the end, you are trying to ‘call’ a tuple, not a function, and ‘that’ is why it fails. :slight_smile:

EDIT: If you want to change your example to actually escaping a function, not ast, then just don’t make it a macro for the go function:

iex(13)> defmodule My do
...(13)>   def go(fun) do
...(13)>     quote do
...(13)>       def hello(), do: unquote(Macro.escape(fun) )()
...(13)>     end
...(13)>   end
...(13)> end
warning: redefining module My (current version defined in memory)
  iex:13

{:module, My,
 <<70, 79, 82, 49, 0, 0, 4, 176, 66, 69, 65, 77, 65, 116, 85, 56, 0, 0, 0, 146,
   0, 0, 0, 17, 9, 69, 108, 105, 120, 105, 114, 46, 77, 121, 8, 95, 95, 105,
   110, 102, 111, 95, 95, 7, 99, 111, 109, ...>>, {:go, 1}}
iex(14)> defmodule Test do
...(14)>   require My
...(14)>   My.go(fn() -> IO.puts "hello" end)
...(14)> end
** (ArgumentError) cannot escape #Function<0.79763921 in file:iex>. The supported values are: lists, tuples, maps, atoms, numbers, bitstrings, PIDs and remote functions in the format &Mod.fun/arity
    (elixir) src/elixir_quote.erl:114: :elixir_quote.argument_error/1
    (elixir) src/elixir_quote.erl:136: :elixir_quote.escape/3
    (elixir) lib/macro.ex:422: Macro.escape/2
    iex:16: My.go/1
    iex:16: (module)

Here it fails because it really can’t serialize a function. :slight_smile:

(Hmm, when were PID’s supported…)

15

You are exactly right: arguments to a macro are automatically converted to ast, so my example doesn’t demonstrate what the docs are trying to say. I posted some examples in one of my earlier posts, which demonstrate that unquote() in fact does not convert to ast.

For a proper example, my example would have to look like this:

defmodule My do
  defmacro go() do
    fun = fn (x) -> x+1 end
    quote do
      def hello(), do: unquote(Macro.escape(fun) )()
    end
  end
end

defmodule Test do
  require My
  My.go()
end

Test.hello

Output:

** (ArgumentError) cannot escape #Function<0.60346940/1 in My.“MACRO-go”/1>. The supported values are: lists, tuples, maps, atoms, numbers, bitstrings, PIDs and remote functions in the format &Mod.fun/arity

…which is the same error that benwilson512 highlighted.

16

Correct, that’s the purpose of unquote, to put something into an AST verbatim. :slight_smile:

17

Yes, I understood that when I started this thread, but I forgot about how macro arguments are automatically converted to ast.

To all, thanks for all the help

1 Like