Why can unquote accept invalid AST in iex?

Questions / Help
1

It seems that IEX is a specially case. I would appreciate some elucidation on what happens in IEX (with respect to AST injection) that allows unquote to accept invalid ASTs. I’d appreciate suggestions on how this feature could be leveraged for testing/scripting as well, if this was the goal.

Context

I read in the docs and in various books/guides that unquote must be passed a valid AST as it injects an AST where unquote is called.

The below is permitted in iex:

1.

iex(91)> quote do unquote %{}
...(91)> end
%{}

2.

iex(92)> quote do: unquote {1, 2, 3, 4, 5}
{1, 2, 3, 4, 5}

3.

iex(93)> quote do
...(93)> first_var_in_block
...(93)> unquote {1, "hello", :pizza, 'world'}
...(93)> third_var_in_block
...(93)> end
{:__block__, [],
 [
   {:first_var_in_block, [], Elixir},
   {1, "hello", :pizza, ~c"world"}, # <= Apparently invalid AST.
   {:third_var_in_block, [], Elixir}
 ]}

Example number three, it can be seen that the evaluation was injected into the midst of the other AST fragments.

Edit:

It’s definitely evalling the arg, I thought it might be skipping it given IEX’s environment,

iex(95)> second_var = 9
9
iex(96)> quote do
...(96)> first_var_in_block
...(96)> unquote {1, "hello", :pizza, second_var}
...(96)> third
...(96)> end
{:__block__, [],
 [
   {:first_var_in_block, [], Elixir},
   {1, "hello", :pizza, 9},
   {:third, [], Elixir}
 ]}
2

I believe it has more to do with that macros must return valid AST. Someone probably has a more succinct answer than me, but iex not causing an error is just that it’s not in a macro.

# foo.ex
defmodule Foo do
  def bar do
    quote do
      unquote({1,2,3,4})
    end
  end

  defmacro baz do
    quote do
      unquote({1,2,3,4})
    end
  end
end

iex> c "foo.ex"
iex> require Foo
iex> Foo.bar()
{1,2,3,4}
iex> Foo.baz()
Invalid AST error
1 Like
3

Ah, this makes sense, since the macro is actually injecting an AST fragment into the overall AST, while the quote is just returning an AST form.

I could have sworn I read that unquote has to be passed a valid AST.

Thanks for clearing that up.

From docs

Unquotes the given expression inside a quoted expression.

This function expects a valid Elixir AST, also known as quoted expression, as argument. If you would like to unquote any value, such as a map or a four-element tuple, you should call Macro.escape/1 before unquoting.
4

It does say that in the docs! I don’t actually know the inner workings there. I always thought of quote and unquote as a smarter string interpolation syntax where unquote is just breaking out of quote. I’m just making making assumptions about how the docs are worded, though.

EDIT: Just caught your edit :slight_smile:

5

I know I ran into an error with invalid AST format a couple of days ago, but I was bouncing back and forth between def and defmacro while experimenting, so it was likely a defmacro error.

It seems that unquote’s prohibtions aren’t really pertinent outside of defmacro, since it doesn’t get used in injection outside of macros.

Yeah that was nail biting :joy:

I’ve seen named functions being defined inside functions using quote and unquote, but I don’t think that qualifies as injection.

6

Ya, I think that’s where the docs are coming from, since there is no use for them outside of macros. Sure you can call them them anywhere you want, but their output is ultimately always going to be used in a macro. The root Web module in Phoenix projects is a good example of that.

If I’m missing something hopefully someone will step in :cold_sweat:

1 Like
7

Given the defmacro and def are macros, there are some interesting code generation possibilities that do not require defmacro. Here’s a very contrived example:

defmodule Q do
  clauses = quote do
    1 -> "Found 1"
    other -> "Found other #{inspect other}"
  end
  
  def myfun(x) when is_integer(x) do
    case x do
      unquote(clauses)
    end
  end
end

Valid AST isn’t always valid executable Elixir because you can quote fragments that, on their own, are not valid Elixir. But they are still valid AST.

5 Likes
8

Yes, I came across this when I was trying to figure out why I could use unquote in a quote do block on a variable in a named function that had not been matched during compile time, but I could not do the same with a “naked” unquote in that same named module definition.

Thankfully I had the recollection that def is a macro so it’s args are quoted meaning that it’s unquote is actually in a nested quote and so we get

iex(97)> quote do
...(97)> quote do
...(97)> unquote some_non_existent_variable
...(97)> end end
{:quote, [context: Elixir],
 [[do: {:unquote, [], [{:some_non_existent_variable, [], Elixir}]}]]}

Rather than an attempt at evaluation for injection, which would take place if the unquote wasn’t in a nested quote.

This means that because unquote’s arg evaluation will take place when the function is called, and not during its AST generation, the AST fragment doesn’t need to be executable at compile time. Gnarly, but full of possibilities (would need heavy documenting for any collaborative efforts involving such an approach).

9

I did not mean in the same named module definition, I meant in the same named function definition. Just caught this typo.