Using unquote outside of quote block

I have always thought that unquote can only be used inside a quote block. Recently i came across this that compiles and works

defmodule Test do 
  x = 10
  def run do
    IO.inspect (unquote(x))
  end   
end

Is this some kind of special allowance by the Elixir compiler?

2 Likes

def is a quote block itself. :smiley:

Ok, so maybe I trivialized that a bit, but the ā€œcallā€ and ā€œexpressionā€ part of a the def macro are quoted parts of the current module.

They are essentially unquoted when they are registered as part of the compilation.

Maybe someone else can explain it better than me.

1 Like

Arguments passed to macro are passed as quoted (meaning they are not evaluated). Since def is a macro, the do block is quoted, and therefore, you can call unquote from it, just like you can from any other argument passed to the macro. The same holds for an invocation of any other macro, including your own:

iex> defmodule Foo do
    defmacro bar(x) do
      IO.inspect x
      nil
    end
  end

iex> require Foo

iex> Foo.bar(unquote(x))
{:unquote, [line: 3], [{:x, [line: 3], nil}]}

iex> Foo.bar do unquote(y) end
[do: {:unquote, [line: 4], [{:y, [line: 4], nil}]}]
9 Likes

Excellent explanation, thank you

This behaviour is called ā€œunquote fragmentsā€ and is documented here: https://hexdocs.pm/elixir/Kernel.SpecialForms.html#quote/2-binding-and-unquote-fragments. The reason it works is because unquote/1 is stored as a function call in the AST like any other function call. Since def/2 is a macro it receives the AST within the :do block and returns a modified AST. It escapes the AST so that it can be returned in its AST form instead of being evaluated. Why itā€™s escaping the code is clear when you think about it, as the module is being defined the code inside def is of course not evaluated, it is only stored so that that it can be compiled after the module and all its functions are defined and then later be evaluated when the function is called.

Since we need to escape the code we can implement unquote fragments because we can simply skip escaping the code inside unquote/1 so that it will be evaluated when the function is defined instead of being escaped and evaluated at runtime. The Macro.escape/1 function can do this for us by passing it unquote: true in the options https://hexdocs.pm/elixir/Macro.html#escape/2.

8 Likes

Excellent answer, thanks for the links

i noticed this phenomenon

iex> defmodule Foo do
    defmacro bar(x) do
      IO.inspect x
    end
  end

iex> require Foo

iex> Foo.bar(unquote(x))
{:unquote, [line: 7], [{:x, [line: 7], nil}]}
** (CompileError) iex:7: unquote called outside quote
    expanding macro: Foo.bar/1
    iex:7: (file)

Why is nil needed? I expected it to work even without nil. Without nil, Foo.bar is returning x as itā€™s quoted expression which is perfectly legit.

Metaprogramming is indeed difficult in the sense that itā€™s so convoluted, there is always a gotcha somewhere.

The result of the macro invocation is an Elixir AST. The shell tries to interpret that ast in its context (because thatā€™s where you invoked the macro). Since the macro is invoked outside of a quote, you get the error.

Hereā€™s an example that works (just paste it all in the shell):

defmodule Foo do
  defmacro bar(x) do
    x
  end
end

defmodule Baz do
  x = 41
  def qux() do
    require Foo
    Foo.bar(1 + unquote(x))
  end
end

Baz.qux()
# 42

The reason why it works is b/c Foo.bar macro is invoked inside a quoted block (do). The macro returns an AST which contains unquote, but thatā€™s fine, because the caller site is itself quoted.

Hope this makes sense :slight_smile:

3 Likes
$ iex
Eshell V8.2  (abort with ^G)
Interactive Elixir (1.4.0) - press Ctrl+C to exit (type h() ENTER for help)
iex(1)> defmodule Foo do
  defmacro bar(x) do
    x
  end
end

defmodule Baz do
  x = 41
  def qux() do
    require Foo
    Foo.bar(1 + unquote(x))
  end
end

Baz.qux()


.
...(1)> * 1: syntax error before: bar
...(1)> ...(1)> .
...(1)> * 1: syntax error before: '.'
...(1)> ...(1)> )
.
...(1)> * 1: syntax error before: ')'
...(1)> ...(1)> :help
.
...(1)> * 1: syntax error before: ':'
...(1)> ...(1)> help.
...(1)> ...(1)> help
...(1)> ...(1)>

Elixir really needs to fix itā€™s shell, Iā€™ve never been able to copy/paste largish chunks of lines into it all at once, yet if I do them one at a time it works fineā€¦ ^.^;

1 Like

You helped report an issue which has been fixed for v1.5. I guess you forgot? :slight_smile:

1 Like

I thought I mentioned in there (or on the commit) that I applied the patch locally to my 1.4 install and it did not work (I still have the patch installed right now with my example above)? Or did further work happen on it that Iā€™ve not seen?

There might be some commits I missed or something, Iā€™ll wait until 1.5 is fully tested and if further issues will report again. :slight_smile:

EDIT: This is my iex.bat currently, the one that does not work still (same issue as my iex session above):

@if defined ELIXIR_CLI_ECHO (@echo on) else  (@echo off)
setlocal
if /I ""%1""==""--help"" goto documentation
if /I ""%1""==""-h""     goto documentation
if /I ""%1""==""/h""     goto documentation
if    ""%1""==""/?""     goto documentation
goto run

:documentation
echo Usage: %~nx0 [options] [.exs file] [data]
echo.
echo  -v                          Prints version and exits
echo  -e COMMAND                  Evaluates the given command (*)
echo  -r FILE                     Requires the given files/patterns (*)
echo  -S SCRIPT                   Finds and executes the given script in PATH
echo  -pr FILE                    Requires the given files/patterns in parallel (*)
echo  -pa PATH                    Prepends the given path to Erlang code path (*)
echo  -pz PATH                    Appends the given path to Erlang code path (*)
echo.
echo  --app APP                   Starts the given app and its dependencies (*)
echo  --cookie COOKIE             Sets a cookie for this distributed node
echo  --detached                  Starts the Erlang VM detached from console
echo  --erl SWITCHES              Switches to be passed down to Erlang (*)
echo  --hidden                    Makes a hidden node
echo  --logger-otp-reports BOOL   Enables or disables OTP reporting
echo  --logger-sasl-reports BOOL  Enables or disables SASL reporting
echo  --name NAME                 Makes and assigns a name to the distributed node
echo  --no-halt                   Does not halt the Erlang VM after execution
echo  --sname NAME                Makes and assigns a short name to the distributed node
echo  --werl                      Uses Erlang's Windows shell GUI (Windows only)
echo.
echo  --dot-iex PATH              Overrides default .iex.exs file and uses path instead;
echo                              path can be empty, then no file will be loaded
echo  --remsh NAME                Connects to a node using a remote shell
echo.
echo ** Options marked with (*) can be given more than once
echo ** Options given after the .exs file or -- are passed down to the executed code
echo ** Options can be passed to the Erlang VM using ELIXIR_ERL_OPTIONS or --erl
goto end

:run
@if defined IEX_WITH_WERL (@set __ELIXIR_IEX_FLAGS=--werl) else (set __ELIXIR_IEX_FLAGS=)
call "%~dp0\elixir.bat" --no-halt --erl "-noshell -user Elixir.IEx.CLI" +iex %__ELIXIR_IEX_FLAGS% %*
:end
endlocal

Are you sure your iex.bat is the one running? I just double checked the issue is fixed on Unix under dumb terminal.

2 Likes

Well I feel stupidā€¦ >.>

Iā€™m using mingwā€™s bash as my shell so it was running iex instead of iex.batā€¦ I hate Windows inconsistencies with everything elseā€¦ >.>

I cannot get it to break, and Iā€™ve tried a dozen things that ā€˜usuallyā€™ do, so it seems fixed. :slight_smile:

/me sighs
Feel free to ignore that commit message. ^.^

It works on my end. Perhaps itā€™s time to switch to mac :stuck_out_tongue:

Forced Windows at work, at home Iā€™m all Linux and do not experience this issue. :wink:

:laughing: i keep forgetting the fact that def is a macro and do block as actually an argument to the macro and hence quoted. Thanks for enlightening me.

1 Like

Sorry to keep harping on this :stuck_out_tongue:
By the same logic, defmodule is a macro, its do block is also quoted
This should work too but doesnā€™t, why?

defmodule Foo do
    unquote(1)
end

Iā€™m not really familiar with compiler internals, but my guess is that technically, defmodule macro is properly invoked. Iā€™d also guess that the macro successfully returns a quoted block, but IEx then tries to evaluate that block, and thatā€™s where things break. I suspect they break b/c during the module compilation the code inside the do block is evaluated (expanded), and at this point we are trying to invoke unquote outside of a macro context (because macro has already returned).

I mostly base my suspicion by looking at the stacktrace of the following iex session:

iex> :erlang.system_flag(:backtrace_depth, 20)

iex> try do 
  defmodule Foo do unquote(1) end 
  catch _,_ -> 
    IO.inspect :erlang.get_stacktrace  
  end

Which gives me:

[{:elixir_exp, :expand, 2, [file: 'src/elixir_exp.erl', line: 10]},
 {:elixir_exp, :expand_block, 4, [file: 'src/elixir_exp.erl', line: 448]},
 {:elixir_exp, :expand, 2, [file: 'src/elixir_exp.erl', line: 39]},
 {:elixir, :quoted_to_erl, 3, [file: 'src/elixir.erl', line: 255]},
 {:elixir_compiler, :code_loading_compilation, 3,
  [file: 'src/elixir_compiler.erl', line: 72]},
 {:elixir_module, :eval_form, 6, [file: 'src/elixir_module.erl', line: 191]},
 {:elixir_module, :do_compile, 5, [file: 'src/elixir_module.erl', line: 71]},
 {:elixir_lexical, :run, 3, [file: 'src/elixir_lexical.erl', line: 17]},
 {:erl_eval, :do_apply, 6, [file: 'erl_eval.erl', line: 670]},
 {:erl_eval, :try_clauses, 8, [file: 'erl_eval.erl', line: 904]},
 {:elixir, :erl_eval, 3, [file: 'src/elixir.erl', line: 224]},
 {:elixir, :eval_forms, 4, [file: 'src/elixir.erl', line: 212]},
 {IEx.Evaluator, :handle_eval, 6, [file: 'lib/iex/evaluator.ex', line: 182]},
 {IEx.Evaluator, :do_eval, 4, [file: 'lib/iex/evaluator.ex', line: 175]},
 {IEx.Evaluator, :eval, 4, [file: 'lib/iex/evaluator.ex', line: 155]},
 {IEx.Evaluator, :loop, 3, [file: 'lib/iex/evaluator.ex', line: 61]},
 {IEx.Evaluator, :init, 4, [file: 'lib/iex/evaluator.ex', line: 21]},
 {:proc_lib, :init_p_do_apply, 3, [file: 'proc_lib.erl', line: 247]}]

To me, that proves that my code is in fact successfully compiled and executed. Otherwise, the try block wouldnā€™t even run, and I wouldnā€™t be able to catch the error. Therefore, the error happens when the quoted block returned by defmodule is interpreted.

But thatā€™s a lot of guesswork on my behalf, so letā€™s ping @josevalim for a precise explanation :slight_smile:

Or i should probably stop spiraling down this rabbit hole :stuck_out_tongue: