Help with dynamic module generation AST

Questions / Help
1

Hello

I’m looking for some help when trying to dynamically generate a module (in this case a Plug router). The code is as follows, but get invalid quoted expression errors when it comes to evaluating the router in the define/2 function.

I’ve tried using some of the Macro functions to produce the correct AST, but to no avail. If anyone can point me in the right direction I would be very grateful!

defmodule Lib.Custom do
  defmacro __using__(_opts) do
      quote do
        @before_compile unquote(__MODULE__)
      end
    end

    defmacro __before_compile__(env) do
      Lib.Custom.Router.define(env, "/hello-world")
    end
  end

defmodule Lib.Custom.Router do
  def define(env, name) do
    router = quote do
      use Plug.Router

      plug :match
      plug :dispatch

      get unquote(name) do
         send_resp(conn, 200, "Hello World")
      end
    end

    env.module
    |> Module.concat(Router)
    |> Module.create(router, line: env.line, file: env.file)
  end
end
2

Your define function does not return a valid quoted expression.

Therefore you need to return something else from your __before_compile__ macro that were a valid quoted expression.

Edit

The created module is only available during compile time as far as I remember, you need to take additional steps to actually persist it.

1 Like
3

If you want to create a module on compile time I think you can just use a macro or return something like this from your before_compile hook:

quote do
  defmodule unquote(modname) do
      ...
  end
end

Your Module.create solution might actually also work, the docs state that it does create a .beam file when it is called during compile time.

5

I figured the bug out at least - it was not returning a valid AST from __before_compile__ as @NobbZ said.
The actual module definition was sound except I still needed to use var!(conn) in the match route.

I also read it not being good practise to define modules in such a manner, but this is a special case.
I learned a little more about Macros at least! :slight_smile:

6

Not sure if said it clearly, so I would like to do it again. In lots of times developer would like to use same module for 2 things (for example using helper library for your library) and then having macro which creates module or function (overriding defmodule, def and defp) are hard to use in some cases.

Here think how developer could create a helper library which adds some extra functions to your generated: MyApp.MyModule.Router - except extra options (like use MyLib.Module, use_modules: [MySecondLib.MyModule] or use MyLib.Module do … end) I don’t see any way to do it for specific module.

Finally as said I prefer:

defmodule MyApp.MyModule.Router do
  use MyLib.MyModule
  use MyHelperLib.MyModule
end

which allows to easily add extra functionality related to your library.

Think that for example HTTPoison is really good, but for example any helper GitHub library which uses HTTPoison is much better as we do not need to worry about changes in GitHub API - just update helper library. Now if you are adding some functions to module then in lots of times somebody would like to write helper function, for example:

defmodule MyApp.Repo do
  use Ecto.Repo, adapter: Ecto.Adapters.Postgres, otp_app: :my_app

  def soft_delete(entry) do
    # use delete function from Ecto.Repo and handle stale error
  end
end

In your case I would need to create 2nd module for helper functions and I would need to ASSUME that it would always have same name (even after library update). Assuming too much things is one of the worst practices I ever made and seen.

Imagine that:

MyApp.MyModule.Router.generated_function()

would accidentally throw compile time errors!

After that think about non-compile time errors, for example calling function dynamically using apply/3. In lots of cases it would be a really big trouble.

So it’s not like creating module is bad practice directly, because in some edge-cases it’s really useful, but it’s not perfect in all use cases as it could later force developer to use true bad practices (like said assuming module name).

It’s also hard to maintain code which requires some assumption for 3rd-party developer. Think that your team joins somebody and would like to introduce American spelling instead of English spelling (or just something like that). This could be reasonable (for your team) change and none of tests would give any error, so other developers could not even notice such breaking backwards compatibility.

From your side such naming change could not be seen like that as you could (again!) assume that you are only one who is using generated code in such module, but again here comes helper library topic.