How to programmatically call macros?

I want to programmatically generate input objects from introspecting my schema, but I can’t figure out the metaprogramming to do so.

Essentially, I want this desired module def:

defmodule InputObjects do
  use Absinthe.Schema.Notation

  input_object :user_filter do
    field :id, :integer_filter
    field :email, :string_filter
  end
end

But from coming from variables:

defmodule InputObjects do
  use Absinthe.Schema.Notation

  name = :user_filter
  fields = [
    {:id, :integer_filter},
    {:email, :string_filter}
  ]
  
  input_object name do
    Enum.each fields, fn {name, type} ->
      field name, type
    end
  end
end

How to do this? Thanks for the help!

@cjbottaro The problem with absinthe is that they are using macros and work only with raw data (not in variables).

defmodule Example do
  defmacro sample(some_data) do
    {:here, [], [:just, :goes, :ast, :without, :quote]}
  end
end

# instead of:

defmodule Example do
  defmacro sample(some_data) do
    quote bind_quoted: [some_data: some_data], unquote: false do
      here(:just, :goes, :normal, :data, :not, :in, :ast, :format, some_data)
    end
  end
end

This is because absinthe is making some checks on raw data.

Let’s say we have such simple code:

defmodule Example do
  defmacro sample(data) do
    IO.inspect(data)
  end
end

Example.sample(5)
5 # IO.inspect call here
5

# vs

data = 5         
5
Example.sample(data)
{:data, [line: 8], nil} # IO.inspect call here
5

As you can see it’s not possible to work on it without proper quoting. Same goes if you want to use absinthe macros i.e. you need to pass raw data.

However this does not mean that it’s not possible to pass variables - this only means that we need pass raw data to absinthe macros. It should be hint for more experienced developers. Just write your own macro!

Firstly you need to know what AST you need to return:

# inside iex call
quote do
  # code of which ast you want to preview
end

# for example:
quote do
  input_object :user_filter do
    field :id, :integer_filter
    field :email, :string_filter
  end
end

{:input_object, [],
 [
   :user_filter,
   [
     do: {:__block__, [],
      [
        {:field, [], [:id, :integer_filter]},
        {:field, [], [:email, :string_filter]}
      ]}
   ]
 ]}

Let’s split it:

  1. {:field, [], [:id, :integer_filter]}
    As you can see it’s field/2 macro AST

  2. {:__block__, [], […]}
    Block here is list of AST expressions inside function which are not single literals. For example: def sample(…) do 5 end gives us just raw 5 in place of whole :__block__ part, but if we add one more line with same literal they would be arguments in :__block__ AST.

  3. Finally {:input_object, [], [:user_filter, [do: …]]}
    Similarly to 1st point it’s ast for input_object/2 call. Here do … end goes to 2nd argument which is keyword list [do: …]. As in 2nd point we could have: [do: 5] or [do: {:__block__, [], […]}]. For us it’s 2nd case as we will never contain literals there.

From this here goes example code:

defmodule Example do
  defmacro sample do
    name = :user_filter

    fields = [
      [:id, :integer_filter],
      [:email, :string_filter]
    ]

    data = Enum.map(fields, &Example.ast_call(:field, &1))
    block = Example.ast_call(:__block__, data)
    do_block_keyword = [do: block]
    Example.ast_call(:input_object, [:user_filter, do_block_keyword])
    # since you generated AST you do not need `quote do … end` here
  end

  # in same way you can simply create helper functions
  # for specific absinthe calls,
  # so you can minimize your initial data (i.e. no need to pass empty list as 2nd argument in each ast_call)
  # and your code is more readable
  def ast_call(name, args), do: {name, [], args}

  # for example:
  # def field_ast(name, type), do: {:field, [], [name, type]}
end

and here is usage:

defmodule InputObjects do
  use Absinthe.Schema.Notation
  require Example
  Example.sample()
end

Sorry if I made any typo - I wrote everything from memory. :077:

2 Likes

Holy moly… thanks for that super detailed explanation!

Let me see if I understand. The strategy here is not to “programmatically call” the Absinthe macros, but rather examine the AST they output and mimic it with my own macro?

Why can the def macro be used with unquote but the Absinthe macros not?

defmodule Foo do
  Enum.each(["foo", "bar"], fn name ->
    def unquote(:"#{name}")() do
      IO.puts unquote(name)
    end
  end)
end

And lastly, does the metadata element of the AST tuple not matter?

Thanks!

I think you just need to use Enum.map/2 instead or use a for comprehension. def has side effects, which is why you can do it in an Enum.each/2 I believe.

Because def is function which can accept any data (raw or in variables). As said absinthe calls like input_object/2 and field/2 can’t use variables as they are macros without quote do … end code (again because of some raw data checks at compile time). Same goes to my version of Example.sample/0 macro. If you would add some arguments then you need to pass only raw data. It’s how meta-programming works.

I recommend to read some articles/books about meta-programming like:

https://www.bignerdranch.com/blog/getting-started-with-elixir-metaprogramming/

because for beginners it’s hard even to visualize how it works.

Ahh, now I see. Ok, yeah in your sample, your macro doesn’t use quote do ... end, got it. :+1:

One last question… how do you make your Example.sample/0 macro take arguments? For example, if you want to pass in the name and fields like so:

defmodule InputObjects do
  use Absinthe.Schema.Notation
  require Example

  name = :user_filter
  fields = [
      [:id, :integer_filter],
      [:email, :string_filter]
    ]

  Example.sample(name, fields)
end

Thanks!

Using Code.eval_quoted/1 seems to work:

defmacro sample(name, fields) do
  {name, _binding} = Code.eval_quoted(name)
  {fields, _binding} = Code.eval_quoted(fields)
end

But I’m unsure if this is the right way.

What I learned in PHP world is to absolutely never use eval unless it’s really, really required, but not sure how it looks like in Elixir. Of course everything depends on some things like from where you are accepting input. If you have data in code then you can simply create it in macro or call some function to return input data.

defmodule Example2 do
  @data …

  def get_data, do: @data
end

defmodule Example do
  defmacro sample do
    input_data = …
    # or:
    input_data = Example2.get_data()
  end
end

Look that when you are accepting any argument in macro then you will receive it quoted. You can use it inside quote do … end or just fetch them from somewhere, but first fetch call needs to come from macro.

defmodule Example2 do
  @file_path "path/to/file.csv"

  def fetch_data_from_csv, do: @file_path |> File.read!() |> CSVLibrary.decode_funcion(…)
end

defmodule Example do
  defmacro sample do
    data = Example2.fetch_data_from_csv()
  end
end

Hi, I just wanted to say thanks for your help. Here’s my final working code; it definitely could not have been done without your help, example code, and explanations.

defmodule Document.InputObjects do
  use Absinthe.Schema.Notation

  input_object :integer_filter do
    field :eq, :integer
    field :gt, :integer
    field :lt, :integer
  end

  input_object :string_filter do
    field :eq, :string
    field :contains, :string
  end

  input_object :datetime_filter do
    field :eq, :string
    field :gt, :string
    field :lt, :string
  end

  import Document.InputObjects.Schema

  input_object_from_schema(Schema.User)
end

defmodule Document.InputObjects.Schema do
  require Logger

  defp filter_name(module) do
    name = module
    |> Module.split
    |> List.delete("Schema")
    |> Enum.join
    |> Macro.underscore
    "#{name}_filter" |> String.to_atom
  end

  defp ecto_type_to_input_object(type) do
    case type do
      :id -> :integer_filter
      :integer -> :integer_filter
      :string -> :string_filter
      :naive_datetime -> :datetime_filter
      :utc_datetime -> :datetime_filter
      type ->
        Logger.warn("No input object defined for type #{inspect(type)}")
        nil
    end
  end

  @doc """
  We're trying to mimic the output of:

  quote do
    input_object :user_filter do
      field :and, list_of(:user_filter)
      field :or, list_of(:user_filter)
      field :not, :user_filter

      field :id, :integer_filter
      field :email, :string_filter
    end
  end

  Which looks like:

  {:input_object,
   [],
   [
     :user_filter,
     [
       do: {:__block__, [],
        [
          {:field,
           [],
           [
             :and,
             {:list_of,
              [],
              [:user_filter]}
           ]},
          {:field,
           [],
           [
             :or,
             {:list_of,
              [],
              [:user_filter]}
           ]},
          {:field,
           [],
           [:not, :user_filter]},
          {:field,
           [],
           [:id, :integer_filter]},
          {:field,
           [],
           [:email, :string_filter]}
        ]}
     ]
   ]}
  """
  defmacro input_object_from_schema(module) do
    {module, _binding} = Code.eval_quoted(module)
    name = filter_name(module)

    fields = [
      [:and, {:list_of, [], [name]}],
      [:or, {:list_of, [], [name]}],
      [:not, name]
    ]

    fields = Enum.reduce module.__schema__(:fields), fields, fn field, acc ->
      ecto_type = module.__schema__(:type, field)
      case ecto_type_to_input_object(ecto_type) do
        nil -> acc
        input_object -> [[field, input_object] | acc]
      end
    end

    block_args = Enum.map fields, fn args ->
      {:field, [], args}
    end

    block_node = {:__block__, [], block_args}

    {:input_object, [], [name, [do: block_node]]}
  end

end

Again, thanks a ton… :pray:

you are passing raw value, but then …

Look that:

quote do
  Schema.User
end

returns:

{:__aliases__, [alias: false], [:Schema, :User]}

so you can do:

defmodule Document.InputObjects.Schema do
  # …

  defp filter_name({:__aliases__, [alias: false], list}) do
    list
    |> List.delete("Schema")
    |> Enum.join()
    |> Macro.underscore()
    |> append("_filter")
    |> String.to_atom()
  end

  defp append(string, suffix), do: string <> suffix

  # …
end

Secondly code is much cleaner when you are writing it top -> bottom instead of bottom -> top. It’s faster to read it.

Finally make sure you are using Elixir builtin code formatter:
https://hexdocs.pm/mix/master/Mix.Tasks.Format.html

I just want to quickly note that one of the goals of Absinthe 1.5 is to need less macros to dynamically define a schema, and I know other people are working on making it simpler as well.

The problem is that people don’t want to write absinthe schema with absinthe macro, but dynamically generate absinthe schema based on ecto database representation + some extra checks and rules.

Oh… well now we’re going to get into a difference of opinions.

so you can do:

defmodule Document.InputObjects.Schema do
  # …

  defp filter_name({:__aliases__, [alias: false], list}) do
    list
    |> List.delete("Schema")
    |> Enum.join()
    |> Macro.underscore()
    |> append("_filter")
    |> String.to_atom()
  end

  defp append(string, suffix), do: string <> suffix

  # …
end

Just because we can do something doesn’t mean we should… or in this case, it’s simpler and/or more readable to eval the args before passing them, instead of passing AST tuples. Also, the arguments are not coming from “user input” so there is no concern with eval’ing them.

I feel like there are two approaches to Elixir metaprogramming:

  • We’re going to help you and hold your hand (and here are the module/funcs to do so)
  • Eff it, here’s the AST and do as you please before sending to the interpreter to execute

So I see what you’re saying: everything can be done at an extremely low level. But if I can do stuff at a higher level, that’s supported by the language’s standard lib. Shouldn’t I do that before just mucking with ASTs?

Firstly AST is not extremely low level if we are talking about Elixir. :077:

Well … I have one really good example from my experience …

My first job … I have worked with Ruby/Rails. I saw really small monkey patching usage from someone who was more experienced than me. I liked how it’s simple (so there was no need to change existing code) and I have stupidly changed file with just few lines of monkey patching into weird file with hundreds of lines with monkey patching.

If something is supported it does not mean that it should be always used. Especially when we are talking about evals. Look … I believe that Joomla! and Wordpress developers are not stupid. Same goes to developers of (at least most popular) extensions for those projects. The problem is that lots of sites was trivially hacked by adding PHP code into image files - just because they used eval.

Sure, one module looks safe, but can you guarantee that this will be still safe after few rotations in dev team? Can you guarantee that it will be still safe after somebody in team decides to move such generator (with eval call) to open source library? Software is changing really rapidly.

If we are going always with easiest solution then we will end with something like Windows in few weeks/months. Windows was created by definitely good developers, but looking how it changes in years we see that something is definitely wrong. For example compare modern (with rewritten code) Plasma 5 features and requirements with what offers Windows's builtin window manager. I can’t do even as good job as that made in both projects, but still I (being not experienced in OS programming) see huge differences.

Also my solution is not about only “low level”. We have here typical pattern matching which prevents to add data of any type. We can always add simple guard to check if module have Schema part in list too without bigger problem.

Summary:

  1. It’s safe - no matter who will use it and no matter if you made mistake
  2. It’s at least as simple as with eval (in implementation)
  3. It’s definitely faster solution

Therefore I’m not sure what’s wrong with my hint.