How to get the AST that would be generated by the given macro?

Suppose we’ve got the following module:

defmodule TestModule do
  defmacro test_macro(param1, param2, param3) do
    quote do
      param1 + param2 + unqoute(param3)
    end
  end
end

How can I get the AST that would be generated by the given macro?

Let’s say that param1 = 1, param2 = 2, param3 = 3
Given those params I’d like to get the following AST:

{:+, [context: Elixir, import: Kernel],
 [
   {:+, [context: Elixir, import: Kernel],
    [{:param1, [], Elixir}, {:param2, [], Elixir}]},
   3
 ]}

I tried to solve this problem with Macro.expand/2 and Code.eval_quoted/3 but wasn’t able to.

Can anyone let me know how to do this or give some hints?

Macros are just functions that take AST as parameters and are expected to return AST. Generally AST is created using quote blocks. But you can quote any code whether in a macro of not. So you to get the AST of the code you show, it would be a matter of:

defmodule TestModule do
  defmacro test_macro(param1, param2, param3) do
    quote do
      param1 + param2 + unqoute(param3)
    end
  end
end

Note that:

  1. In this case param1 and param2 in the quote block have nothing to do with the parameters passed to the macro. They are variable references that would need to be referenceable in the calling site.
  2. unquote will dereference the parameter param3. If your intent was to get the AST for unquote(param3) then change the quote do block to be quote unquote: false do
1 Like

Do note that due to macro hygene, simply having param1 as a bound variable at the call site will actually not work without opting out of hygene with var!

@bartblast you can see the AST by putting an IO.inspect after your quote do block

defmodule TestModule do
  defmacro test_macro(param1, param2, param3) do
    param1 + param2 + unqoute(param3)
  end
  |> IO.inspect
end
1 Like

@kip, @benwilson512
Sorry, I forgot about wrapping the code inside the macro with quote, but that was my intent (I edited the module code).

I need to get the AST programmatically without touching the current code.
So, I’d like to implement the following function that can be called anywhere:

defmodule Helpers do
  def get_macro_generated_ast(module, function, args) do
    …
  end
end

And calling this function like this:
Helpers.get_macro_generated_ast(TestModule, :test_macro, [1, 2, 3])
Would return the AST which i specified in my first post.

What you are looking for is Macro.expand_once/2 or Macro.expand/2 if you want to expand it fully. Just beware, that it is no safer than Code.eval_quoted/3, so do not use it on untrusted input.

4 Likes

@hauleth
That’s what I did initially, but there are 2 problems with this:

Before running:

Macro.expand((quote do: TestModule.test_macro(1, 2, 3)), __ENV__)

I need to explicitely require the TestModule.

the result AST contains an “unquote” expression instead of a value:

{:+, [context: TestModule, import: Kernel],
 [
   {:+, [context: TestModule, import: Kernel],
    [
      {:param1, [counter: -576460752303422365], TestModule},
      {:param2, [counter: -576460752303422365], TestModule}
    ]},
   {:unqoute, [], [{:param3, [counter: -576460752303422365], TestModule}]}
 ]}

Yes, because require mean “ensure that given module is compiled”. Module must be compiled to be able to use macros from it.

1 Like

Ouch… it looks like it’s not possible to do with the standard lib, then. I’ll have to develop some custom solution for this. Thank you all for help!

You can always use my library:

test.ex

defmodule TestModule do
  defmacro test_macro(param1, param2, param3) do
    p1_and_p2 = param1 + param2
    quote do
      unquote(p1_and_p2) + unquote(param3)
    end
  end

  def testing do
    test_macro(1, 2, 3)
  end
end

elixirc text.ex
mix run run.exs --beam Elixir.TestModule.beam

defmodule TestModule do
  def(MACRO-test_macro(&("CALLER"), _param1, _param2, _param3)) do
    _p1_and_p2 = _param1 + _param2
    {:+, [context: TestModule, import: Kernel], [_p1_and_p2, _param3]}
  end
  def testing() do
    3 + 3
  end
end
1 Like

What exactly you want?

1 Like

Do you mean what I need this code for?

Yes, because now it sounds like XY problem.

@hauleth
I’m writing Elixir to JavaScript transpiler as part of a project I’m working on. When I transpile a specific Elixir module I need to expand all the macros and use directives before transforming the AST to IR. If such macro expansion can’t be done with Elixir standard lib, I’m still able to get the macros AST, resolve the bindings and inject the generated AST into the calling module, but this would be more time-consuming with my custom approach, so I hoped I could do this differently, just with the standard lib somehow.

If you are doing an elixir to js transpiler, I recommend using the BEAM bytecode. You can obtain bytecode by doing :beam_disasm on the module binary.

6 Likes

@ityonemo
Actually, I tried this approach initially, but Elixir AST is much more understandable to me and very straightforward to work with.

what’s wrong with using binding() |> IO.inspect? That’s what I was just doing trying to learn what ASTs looked like in the Elixir macro world.

@stevensonmt
Can you explain what do you mean? binding/1 returns the variable name-value bindings for the current context.

If you pass the AST to a helper function you can use binding/1.

@stevensonmt
Could you give some simple example using the module which I described in my first post?

the problem is that macros are HIGHLY contextual and the “middle” of the macro code (everything between the defmacro and the resulting quoted retval), may not be able to evaluated without having access to a bunch of internal information, if for example, functions like Module.get_attribute (see note “This function can only be used on modules that have not yet been compiled”) and then if you have a macro inside of a macro, you will need to track the rolling environment, and then there’s the magic __CONTEXT__ special form, which quite frankly I’m not entirely sure how it works under the hood. I guess that’s why you’re trying to obtain the AST information at the point of egress, but note that even the AST emitted by module ITSELF might have macros inside of it, so even if you obtain the AST in a late-interception fashion, you still need something that can “execute” macros.

I think this complexity might have been what killed ElixirScript – elixir_script v0.32.1, which I have a ton of respect for. If you decide to go with reading bytecode, which honestly is IMO way simpler than elixir AST, let me know, I am very much intereseted in resurrecting these elixir on the FE type projects. I am thinking about tinkering with a WASM-based bytecode interpreter, (not to be confused with lumen, which is a beam-to-wasm compiler).

2 Likes