Unexpected generated module name when playing with AST

Hey, guys. I have a question about the metaprogramming.

Here is an example of my code (the online version):

defmodule MiniParams do
  defmacro defparams(name, do: block) when is_atom(name) do
    module_name = Module.concat([__MODULE__, Macro.camelize("#{name}")])

    quote do
      defmodule unquote(module_name) do
        def echo() do
          unquote(block)
          IO.inspect(__MODULE__)
        end
      end
      
      # other code will operated on above generated module
      # ...
    end
  end
end

defmodule ControllerA do
  require MiniParams

  MiniParams.defparams :search do
    IO.inspect("search defined in ControllerA")
  end
end

defmodule ControllerB do
  require MiniParams

  MiniParams.defparams :search do
    IO.inspect("search defined in ControllerB")
  end
end

After calling defparams in ControllerA and ControllerB, I expect that two modules will be generated:

ControllerA.MiniParams.Search
ControllerB.MiniParams.Search

But, currently, they all generate:

MiniParams.Search

And, the former one will override the previous one.


In order to make it work as I expected, I have to change the code (the online version):

# from
module_name = Module.concat([__MODULE__, Macro.camelize("#{name}")])
# to (specify the full name of module)
module_name = Module.concat([__CALLER__.module, __MODULE__, Macro.camelize("#{name}")])

But, I’m wondering why the original code doesn’t work ?


I did some experiments about this. If I don’t make module dynamic, everything is fine (the online version):

defmodule MiniParams do
  defmacro defparams(name, do: block) when is_atom(name) do
    quote do
      # hardcode the expected module name
      defmodule MiniParams.Search do
        def echo() do
          unquote(block)
          IO.inspect(__MODULE__)
        end
      end
      
      # other code will operated on above generated module
      # ...
    end
  end
end

defmodule ControllerA do
  require MiniParams

  MiniParams.defparams :search do
    IO.inspect("search defined in ControllerA")
  end
end

defmodule ControllerB do
  require MiniParams

  MiniParams.defparams :search do
    IO.inspect("search defined in ControllerB")
  end
end

Back to question again, why does the original code not work?

1 Like

Hello @c4710n

This code works as you expecting:

defmodule MiniParams do
  defmacro defparams(name, do: block) do
    module = inspect(__MODULE__)
    module_name = "#{module}.#{Macro.camelize(to_string(name))}"

    quote do
      defmodule :"#{(__MODULE__)}.#{unquote(module_name)}" do
        def echo() do
          unquote(block)
          IO.inspect(__MODULE__)
        end
      end

      # other code will operated on above generated module
      # ...
    end
  end
end

defmodule ControllerA do
  require MiniParams

  MiniParams.defparams :search do
    IO.inspect("search defined in ControllerA")
  end
end

defmodule ControllerB do
  require MiniParams

  MiniParams.defparams :search do
    IO.inspect("search defined in ControllerB")
  end
end

The problem is how you are construct the module_name. In your code you gets two times Elixir.MiniParams.Search because your __MODULE__ for modules_name refers to MiniParams. I assume that the Elixir. namespace prevents that the controller from generating a nested module. Maybe someone else has a better explanation. One addition, every elixir module lives in the Elixir namespace, but that namespace is hidden most of the time.

Edit: Here is one more example:

defmodule Foo do
  defmodule Elixir.Bar do
    def bar, do: :bar
  end

  defmodule Baz do
    def baz, do: :baz
  end

  def foo, do: :foo
end
iex(1)> Foo.foo()
:foo
iex(2)> Bar.bar()
:bar
iex(3)> Baz.baz()
** (UndefinedFunctionError) function Baz.baz/0 is undefined (module Baz is not available)
    Baz.baz()
    iex:3: (file)
iex(3)> Foo.Baz.baz()
:baz
iex(4)> Foo.Bar.bar()
** (UndefinedFunctionError) function Foo.Bar.bar/0 is undefined (module Foo.Bar is not available)
    Foo.Bar.bar()
    iex:4: (file)

Edit: This content is overrided accidently. ;(

Check out my next post.

That is wrong. It seems that this is equivalent but it isn’t.

defmodule Foobar do
  def test do
    IO.inspect(__MODULE__, label: :a)
    IO.puts("b: #{__MODULE__}")
    module = inspect(__MODULE__)
    IO.inspect(module, label: :c)
    IO.puts("d: #{module}")

    name = "Search"

    module_name = "#{module}.#{name}"
    IO.inspect(module_name, label: :module_name)
    IO.puts("module_name: #{module_name}")

    module_concat = Module.concat([module, name])
    IO.inspect(module_concat, label: :module_concat)
    IO.puts("module_concat: #{module_concat}")
  end
end
iex(1)> Foobar.test
a: Foobar
b: Elixir.Foobar
c: "Foobar"
d: Foobar
module_name: "Foobar.Search"
module_name: Foobar.Search
module_concat: Foobar.Search
module_concat: Elixir.Foobar.Search

So, if your macro returns defmodule Elixir.SomeModule ... the module will be generated in the Elixir namespace when it returns defmodule SomeModule it will work as nested module. In my code the generated module name is Elixir.ControllerA.MiniParams.Search.

Edit: :thinking: - My last sentence isn’t 100% correct. I think there is one point I am missing.

1 Like

@Marcus Thanks you for the explanation. I think I got your points:

  • Module names are different from plain atoms. For example:
    • Thing is a module name. Internally, it’s an atom - :"Elixir.Thing".
    • :Thing is a plain atom.
iex> Thing == :Thing
false

iex> Thing == :"Elixir.Thing"
true
  • A module name can be converted to an atom which is not prefixed by Elixir:
iex> :"#{inspect(Thing)}" == :Thing
true
  • Module.concat returns a module name.
  • When passing a module name to defmodule unquote(module_name), such as MiniParams.Search, the module being defined is Elixir.MiniParams.Search.

Above facts clears my confusions.


To emulate the behavior of nested modules, I also explored the quote expressions generated by different code.

Let’s see the code with problem:

defmodule MiniParams do
  defmacro defparams(name, do: block) when is_atom(name) do
    module_name = Module.concat([__MODULE__, Macro.camelize("#{name}")])

    ast =
      quote do
        defmodule unquote(module_name) do  # notice this line
          def echo() do
            unquote(block)
            IO.inspect(__MODULE__)
          end
        end

        # other code will operated on above generated module
        # ...
      end

    IO.inspect(ast)

    ast
  end
end

defmodule ControllerA do
  require MiniParams

  MiniParams.defparams :search do
    IO.inspect("search defined in ControllerA")
  end
end

Above code generates the following AST:

{:defmodule, [context: MiniParams, import: Kernel],
 [
   MiniParams.Search,  # notice this line.
   [
     do: {:def, [context: MiniParams, import: Kernel],
      [
        {:echo, [context: MiniParams], []},
        [
          do: {:__block__, [],
           [
             {{:., [line: 7], [{:__aliases__, [line: 7], [:IO]}, :inspect]},
              [line: 7], ["search defined in ControllerA"]},
             {{:., [], [{:__aliases__, [alias: false], [:IO]}, :inspect]}, [],
              [{:__MODULE__, [if_undefined: :apply], MiniParams}]}
           ]}
        ]
      ]}
   ]
 ]}

Next, let’s see the code without problem:

defmodule MiniParams do
  defmacro defparams(name, do: block) when is_atom(name) do
    module_name = Module.concat([__MODULE__, Macro.camelize("#{name}")])

    ast =
      quote do
        defmodule MiniParams.Search do   # notice this line, I hardcode the module name.
          def echo() do
            unquote(block)
            IO.inspect(__MODULE__)
          end
        end

        # other code will operated on above generated module
        # ...
      end

    IO.inspect(ast)

    ast
  end
end

defmodule ControllerA do
  require MiniParams

  MiniParams.defparams :search do
    IO.inspect("search defined in ControllerA")
  end
end

Above code generates the following AST:

{:defmodule, [context: MiniParams, import: Kernel],
 [
   {:__aliases__, [alias: false], [:MiniParams, :Search]}, # notice this line
   [
     do: {:def, [context: MiniParams, import: Kernel],
      [
        {:echo, [context: MiniParams], []},
        [
          do: {:__block__, [],
           [
             {{:., [line: 7], [{:__aliases__, [line: 7], [:IO]}, :inspect]},
              [line: 7], ["search defined in ControllerA"]},
             {{:., [], [{:__aliases__, [alias: false], [:IO]}, :inspect]}, [],
              [{:__MODULE__, [if_undefined: :apply], MiniParams}]}
           ]}
        ]
      ]}
   ]
 ]}

Diff the generated AST generated by two pieces of code, we can find the difference (which are marked by the comment #notice this line):

AST #1: MiniParams.Search
AST #2: {:__aliases__, [alias: false], [:MiniParams, :Search]}

If I want to emulate the behavior of nested modules like:

defmodule A do
  defmodule B do
  end
end

I have to generate AST like AST #2.


In my case, a possible solution is:

defmodule MiniParams do
  defmacro defparams(name, do: block) when is_atom(name) do
    aliases = {
      :__aliases__,
      [alias: false],
      [:"#{inspect(__MODULE__)}", :"#{Macro.camelize(to_string(name))}"]
    }

    quote do
      defmodule unquote(aliases) do
        def echo() do
          unquote(block)
          IO.inspect(__MODULE__)
        end
      end
    end
  end
end

defmodule ControllerA do
  require MiniParams

  MiniParams.defparams :search do
    IO.inspect("search defined in ControllerA")
  end
end

defmodule ControllerB do
  require MiniParams

  MiniParams.defparams :search do
    IO.inspect("search defined in ControllerB")
  end
end

Although above code and @Marcus code work, but I still prefer my original way to generate module_name:

module_name = Module.concat([__CALLER__.module, __MODULE__, Macro.camelize("#{name}")])

Because it uses the public API of Elixir, without depending the internal machenism.

2 Likes

Solutions:

This is a letter to our future friends. :wink:

2 Likes