Escaping anonymous function in module attribute

Questions / Help
,
1

I’m currently writing a specie of DSL and I’m trying to understand how can I escape an anonymous function in a module attribute. I’m not sure how it can be achieved and even tried without success to understand how Absinthe does it with the resolve macro (e.g. field :name, :string, resolve: _, _ -> ... end [Absinthe.Type.Field — absinthe v1.7.6]). This is what I have tried so far:

defmodule DSL do
  defmacro __using__(_opts) do
    quote do
      import DSL, only: :macros

      Module.register_attribute(__MODULE__, :columns, accumulate: true)

      @before_compile DSL
    end
  end

  defmacro __before_compile__(_env) do
    quote do
      def __columns__, do: Enum.reverse(@columns)
    end
  end

  defmacro column(field, resolver) do
    quote do
      @columns {unquote(field), unquote(resolver)}
    end
  end
end

defmodule User do
  use DSL

  defstruct ~w[first_name last_name email]a

  column :name, fn user -> "#{user.first_name} #{user.last_name}" end
  column :email, fn user -> user.email end

  def test do
    user = %__MODULE__{first_name: "John", last_name: "Silva"}
    columns = __MODULE__.__columns__()

    then(user, columns[:name])
  end
end

which fails with:

** (ArgumentError) cannot inject attribute @columns into function/macro because cannot escape #Function<1.54200728/1 in :elixir_compiler_0.__MODULE__/1>. The supported values are: lists, tuples, maps, atoms, numbers, bitstrings, PIDs and remote functions in the format &Mod.fun/arity
    (elixir 1.16.0) lib/kernel.ex:3648: Kernel.do_at/5
    (elixir 1.16.0) expanding macro: Kernel.@/1
    /home/wigny/workspace/dsl.exs:25: User.__columns__/0
    /home/wigny/workspace/dsl.exs:25: DSL.__before_compile__/1

I’m aware I could simply solve it by replacing the anonymous function call with a captured function call, but I would like to understand how libraries like Absinthe solve this problem…

and Happy New Year to everybody :).

2

I cannot overemphasize how much I regret allowing this in Absinthe :D. :persistent_term sort of saves my bacon but that required rewriting the whole DSL into something that also operates at runtime which is sort of worth it but now we’re into a whole OTHER discussion of runtime vs compile time DSLs.

Supporting compile time anonymous function is a small nightmare. In the compile time version of Absinthe schemas where they’re compiled to a module body the answer simply is: I don’t. The AST of the anonymous function gets “lifted” up and then dropped into the function body of a __absinthe_function__ clause so that they can be fetched at runtime as part of middleware. In the :persistent_term version I think they get inlined? Honestly I haven’t looked at the details in a while.

Here’s why they’re a nightmare. Let’s take your nice simple example:

  column :name, fn user -> "#{user.first_name} #{user.last_name}" end
  column :email, fn user -> user.email end

And add stuff that as the end user you think should work:

  @prefix :foo
  column :name, fn user -> "#{@prefix} #{user.first_name} #{user.last_name}" end
  @prefix :bar
  column :email, fn user -> "#{@prefix} #{user.email}" end

OR

  prefix =  :foo
  column :name, fn user -> "#{prefix} #{user.first_name} #{user.last_name}" end
  prefix  = :bar
  column :email, fn user -> "#{prefix} #{user.email}" end

OR

  column :name, fn user -> "#{Application.get_env(:my_app, :prefix)} #{user.first_name} #{user.last_name}" end

etc.

These get horrendously tricky to implement because module attributes haven’t actually been executed at the point that your macro is firing, they’ve only become AST itself, and other details depend wildly on how your store your function. It’s even possible to store functions that can’t be used at runtime at all!

Absinthe was built before Persistent term existed. If I could do it all over I’d have a purely functional DSL that just made a nice data structure, and then I’d require that you:

children = [
  ... # ecto goes here
  {Absinthe.Schema, your_schema_builder_result_here},
  # ... Other stuff like phoenix
]

and Absinthe would consume the data stucture emitted by the functional DSL and put whatever structures into :persistent_term it deemed efficient. For Absinthe 2.0 (no immediate plans or timeline) that is exactly what I’ll do. Even for a lot of things like resolvers though it’s just way clearer to point to a “regular” function via &Module.function/3 because there’s no closure over a complicated and often difficult to determine environment.

This is all of a very long winded way of saying: I don’t recommend this style of DSL. Strongly consider other options to avoid it. cc @zachdaniel since he’s done a bunch of similar stuff over in Ash that has at this point far surpassed what I did DSL wise in Elixir.

3 Likes
3

Honestly we’ve pretty much “figured it out” with our DSL builder Spark. It escapes functions just fine by defining a named function in the module and referencing that. If you have a DSL option type that is a function, this all gets handled for you automagically. You can also use a higher level type {:spark_function_behaviour, behaviour, {function_mod, arity}} that a DSL option can adopt. In that case, you can provide a behaviour and opts pair like @benwilson512 illustrates, or you can provide an anonymous function of a given arity. And it defines a function in the module and references that, passing it to {TheMod, fun: TheFun}.

Spark is still sorely undocumented, but it’s heavily battle tested and has good tests that you can learn from :slight_smile: you could build all of absinthe’s DSL or phoenix router in spark with the tools it has today. And we provide an elixir_sense plugin that provides autocomplete with in-line DSL docs with no action required at all.

Edit: link to spark Get Started With Spark — spark v1.1.53

2 Likes
4

How does it handle the cases where the anonymous function references values “outside” the function body?

Overall though that is very exciting, and I’m delighted to see that you made it a separate library. Great work!

5

We define the function where the value is in the AST, so captures “just work”. Given a value that may contain an anonymous function, this function returns new AST to reference a named function, and a quoted expression that will define those functions. The function name is derived from the option name and a hash of the code.

So to the rest of the code it looks like you’re always referencing &Module.fun/arity so it’s opaque.

EDIT: We also use a monotonically increasing number per module, which TBH may allow us not to bother with using the hash of the code, which would make for much prettier function names. I’ll look into it. Ultimately its all opaque to the user, except for that function name potentially appearing in stack traces (but with correct line numbers pointing to the anonymous function)