Defining functions in a loop inside a macro

Questions / Help
1

I can dynamically define functions in a “loop” like this:

defmodule Foo do
  for {f, i} <- [{:foo, "foo"}, {:bar, "bar"}] do
    def foo(unquote(f)) do
      unquote(i)
    end
  end
end

iex> Foo.foo(:foo)
"foo"
iex> Foo.foo(:bar)
"bar"
iex> Foo.foo(:baz)
💣

So that’s all cool :+1:

Now how do I do this inside of a macro? I’ve been at this for a few hours now and having a very hard time wrapping my head around it. The closest I’ve come is this:

defmodule Foo do
  defmacro __using__(_) do
    quote do
      for {f, i} <- [{:foo, "foo"}, {:bar, "bar"}] do
        defmacro foo(f) do
          quote do
            # unquote(i)  <- This doesn't work so I'm commenting it out for now
          end
        end
      end
    end
  end
end

This has the effect of defining def foo(:foo) twice, even though I dbg(f) in inside the inner macro and it has the correct values. On top of that, I can’t access i inside the inner macro.

I got a bit of hint from this thread. I tried re-quoting that whole comprehension, capturing it in a variable, then quote(do: unquote_splicing(block)) it, but that didn’t work. There is more code that follows it so, I dunno.

I appreciate any help though if this serves as a rubber duck that’d be good too :slight_smile:

1 Like
2

It’s worth remembering that the return value of a macro just needs to be a valid AST data structure to be inserted somewhere else, which can be composed however you like, including raw literals! It’s easy to forget sometimes, because quote is so powerful, and I often fall into this block-based thinking where the final expression of a defmacro has to be a top-level quote block.

However, an equally valid return value is a list of AST snippets, which will get interpreted as a series of sequential expressions. So you can just tuck the quote inside the for, and return a list of quoted expressions:

defmodule Foo do
  defmacro __using__(_opts \\ []) do
    for {function_argument, function_return} <- [{:fizz, "fizz"}, {:buzz, "buzz"}] do
      quote do
        def baz(unquote(function_argument)) do
          unquote(function_return)
        end
      end
    end
  end
end

defmodule Bar do
  use Foo
end

Bar.baz(:fizz)
#=> "fizz"
4 Likes
3

Hey Chris! So had I tried something like this but of course I didn’t give the whole picture in my post. I’m iterating over protocol impls, so they need to be retrieved in a quote block in order for it to be consolidated (as far as I can tell, at least). However, you’ve definitely given me a good hint here. I assume I’m going to have to iterate over ast and inject it. Something like:

defmacro __using__(_) do
  {_, _, result_ast} =
    quote do
      {:consolidated, mods} = P.__protocol__(:impls)

      for mod <- mods do
        type = mod.__struct__
        {P.name(type), P.body(type)}
      end
    end

  result =
    for ast <- result_ast do
      quote do
        # Profit
      end
    end

  quote do
    unquote(result)
  end
end

Something like that? In any event, you’ve given me a good hint for a way forward. Thanks!!

I’ll report back if I get this working.

1 Like
4

Ah yeah, that’ll be tricky, based on when protocols get consolidated during the compilation process. Definitely interested to learn if you solve it!

5

I am quickly realizing this :sweat_smile: I think I’m going to explore other options.

EDIT: Ya, re-reading the docs they are consolidated after everything else is compiled, so what I want to do is not possible. I have a runtime version of it working so I will probably just stick with that for now.

Thanks again for your help!

2 Likes
6

When you have nested quote blocks, you need to give Elixir a hint as to what an unuquote applies to. You do this by passing quote unquote: false to the outer block. The following compiles, but I haven’t tested if its actually what you want:

iex(1)> defmodule Foo do
...(1)>   defmacro __using__(_) do
...(1)>     quote unquote: false do
...(1)>       for {f, i} <- [{:foo, "foo"}, {:bar, "bar"}] do
...(1)>         defmacro foo(f) do
...(1)>           quote do
...(1)>             unquote(i)
...(1)>           end
...(1)>         end
...(1)>       end
...(1)>     end
...(1)>   end
...(1)> end
{:module, Foo,
 <<70, 79, 82, 49, 0, 0, 6, 168, 66, 69, 65, 77, 65, 116, 85, 56, 0, 0, 0, 177,
   0, 0, 0, 17, 10, 69, 108, 105, 120, 105, 114, 46, 70, 111, 111, 8, 95, 95,
   105, 110, 102, 111, 95, 95, 10, 97, 116, ...>>, {:__using__, 1}}
6 Likes
7

Whoa! With a slight tweak that worked! Thanks so much, Kip! Though now I’m puzzled as to how it works with protocol impls, though the docs do qualify that they are usually consolidated after the app is compiled. I’m guessing if there is a compile-time dependency on it that will make it happen earlier. That is maybe supported by this but I haven’t taken the time to grok it yet.

Thanks again!

defmacro __using__(_) do
  func =
    quote unquote: false do
      {_, mods} = Proto.__protocol__(:impls)

      for mod <- mods do
        type = mod.__struct__
        name = Proto.name(type)
        ast = Proto.body(type)

        def func(unquote(name)) do
          unquote(ast)
        end
      end
    end

  quote do
    unquote(func)

    # more stuff that depends on func/1
  end
end
2 Likes
8

I suspect it’s just a race condition in the compiler, and beating it is a matter of project size.

Not to X Y problem you, but why are you trying to metaprogram generated functions based on structs implementing protocols? I ask for three reasons,

  1. Not having to know in advance at compile time what modules offer an implementation is part of the point of protocols, so I am curious as to your usecase
  2. I am even more curious to your usecase, as I like committing macro crimes
  3. I know of some macro crimes that may help, depending on the Y of your problem
1 Like
9

This is very unsurprising to hear. I had an inkling that I was ignoring for now while I reveled in this working. I feels incredibly janky, of course!

I don’t mind being XY’d, I do it to people on this site but ya, I guess they sometimes haven’t appreciated it :sweat_smile: I’m just playing around building a little test factory lib. I’m looking for a way to define factory definitions across multiple files with little ceremony, so protocols came to mind. Of course, I don’t want to do build(MyApp.Context.Entity), I want to do build(:entity), so then I got this idea of writing a macro that stores the key in the protocol implementations then do roughly what I shared up there :point_up: And now it works, but only in my very small tests, of course! I just got fixated on making that particular solution work but there’s probably a simpler way? I’m happy to hear of your crimes!

I know I can also just a process or ETS for this, the game was more to see if I could avoid it.

1 Like
10

I missed this response til now, happy to oblige! :smiling_imp:

There’s no real way to do this at compile time as a library AFAICT, since to be race-condition-proof, you have to assume your library modules must always compile before the application code that would register its own modules under these aliases with your library module—your modules are not “open” for modification in the Ruby sense.

You have more options if you are willing to write macros in your namespace (ex Factory) that help application code define its own module within its own namespace (ex App.Factory), however.

Based off of my uses tracker, you could have an API like:

defmodule App.Factory do
  use Factory.Builder

  register App.Foo, as: :foo
  register App.Bar, as: :bar
end

App.Factory.build(:foo)
#=> %App.Foo{fizz: nil}

Of course, at this point you may as well use compile-time Config. However, if you’d rather the :as configuration be defined adjacent to the definition of the struct modules themselves, like so:

defmodule App.Foo do
  use Factory.Generator, as: :foo
  defstruct [:fizz]
end

defmodule App.Factory do
  use Factory.Builder

  register [App.Foo, App.Bar]
end

…then you have to get a little fancier, but it can definitely be done. This approach is overkill, but convenient if you’re deadset on this approach anyways and Factory.Generator has other work it wants to do in the struct module already.

1 Like
11

I thought about something along the lines of the builder you suggested, but my goal was to either make it dead-simple or not bother at all. I think I’m going to not bother at all :smiley: I’m mostly just experimenting on making some small QOL improvements on Ecto’s test factories, which I like very much and are all I’ve ever used in my personal projects. I’ll take a closer look at your uses tracker and maybe get inspired!

Thanks for all the info, it’s much appreciated!

1 Like