Module Does Not Recompile When Referenced Module Changes

Hi folks, got a question about how Elixir determines when to recompile a module. The documentation says a module will recompile when one of its dependencies changes, but that doesn’t seem to be the case here and I’m not sure why. Here’s the scenario:

I have a module that defines a schema for the library I’m building:

defmodule Schema do
  import Library.Schema.Notation

  schema :name do
    # macros and schema definition...
  end
end

When this compiles, it exposes a callback __schema__ to be consumed in another module. While Library.Client actually consumes the schema, Testbed is the intended product module:

defmodule Testbed do
  use Library.Client, schema: Schema
  #. (Library.Client.__using__() actually calls Schema.__schema__())
  # ... callbacks and implementation
end

I figured because Testbed references Schema, then it should know to recompile if Schema recompiles. I’m consistently seeing however that changes made to Schema do cause itself to recompile, but Testbed still doesn’t and I have to manually recompile it to reflect the changes in Schema. This ring a bell for anyone?

Thanks,
Mike

1 Like

With an oversimplified mix project, I am unable to reproduce this:

defmodule Recompile.Schema do
  defmacro __schema__() do
    quote do
      "I do schema stuff!"
    end
  end
end

defmodule Recompile.Notation do
  defmacro __using__([schema: schema]) do
    quote do
      require unquote(schema)
      unquote(schema).__schema__()
    end
  end
end

defmodule Recompile.TestBed do
  use Recompile.Notation, schema: Recompile.Schema
end
$ mix xref graph --label compile --sink lib/recompile/schema.ex
lib/recompile/test_bed.ex
└── lib/recompile/schema.ex (compile)

and when I change Schema:

$ mix compile --verbose
Compiling 2 files (.ex)
Compiled lib/recompile/schema.ex
Compiled lib/recompile/test_bed.ex

Could you possibly share more code and your Elixir version? Is this in a mix project?

2 Likes

I’m able to reproduce it with a __schema__() function (not macro) and without the require unquote(schema).

Think the issue might be with the fact that schema in unquote(schema).__schema__() is dynamic and there is no hard dependency on the module, in might be loaded at runtime after all. Adding Code.ensure_compiled!(unquote(schema)) inside the quote block in defmacro __using__(..) works for me.

defmodule Schema do
  def __schema__(), do: :ok
end

defmodule Client do
  defmacro __using__(schema: schema) do
    quote do
      Code.ensure_compiled!(unquote(schema))
      # ...
    end
  end
end

defmodule Testbed do
  use Client, schema: Schema
end

My first attempt was with a non-macro __schema__() and no require and still couldn’t reproduce. I just changed to a macro to more closely fit what I imagine the real code is.

Ah ya, in your example simply doing unquote(schema) isn’t going to do anything anything meaningful if you’re just using a regular function in the way you have it. You’d need to do import unquote(schema) for __schema__ to be available in Testbed.

Yeah, I’m guessing the call to __schema__() is itself inside a function definition within the macro’s quote block? If it’s called outside as in your example, it works for me, too.

It’s just that saying unquote(schema) on its own is going to expand to just Schema which is simply the atom Schema which is a no op. If you do import unquote(schema) that will actually import its functions. You don’t need ensure_compiled there as import ensures its compilation.

That’s true, I assumed @mjlorenzo’s Library.Client module would look something like this:

defmodule Client do
  defmacro __using__(schema: schema) do
    quote do
      # Code.ensure_compiled!(unquote(schema))
      def my_schema(), do: unquote(schema).__schema__()
    end
  end
end

In that case, without Code.ensure_compiled!, Testbed is not recompiled when Schema changes.

Yes, that is correct behaviour in this situation because it’s inside a function which makes it a runtime dependency.

1 Like

@sodapopcan, @awerment thanks y’all for the discussion. This is the top of Library.Client.__using__:

 defmacro __using__(opts) do
   raise_if_no_schema(opts)

   schema = opts[:schema]
   module = Macro.expand(schema, __ENV__)
   Code.ensure_compiled!(module)
   defs = module.__schema__()

If i’m understanding correctly, because the above is entirely in the macro body of Library.Client.__using__, Testbed itself has no idea to interpret Schema as anything other than the atom and so no dependency exists. Injecting some sort of import, require, etc into Testbed will establish this. On the right track?

Mostly, yes. Nothing outside of the returned quoted expression will be injected into Testbed so once that module is compiled, all that stuff is gone. So in this case it’s not that it isn’t interpreting Schema as anything other than the atom it’s that Schema isn’t even present in the compiled code! Again it’s hard to see without more context but there doesn’t seem to be anything about that code snippet that suggests you need Macro.expand over requireing or importing within a quote block which of course you want anyway to create the dependency. It depends what you’re trying to accomplish, of course.

I’ve also never actually seen ensure_compiled! used inside a module before. I’m not a master of macros, though, so maybe there is a good reason, but AFAIK, ensure_compiled! is essentially require for use outside of a defmodule. Don’t quote me on that, though :slight_smile:

1 Like

As @sodapopcan wrote, having all those references to the schema module outside of the quote block means that no compile-time dependency to it is established for the Testbed module itself. If you need to import/require functions/macros from it in Testbed, do so inside the quote block, otherwise having ensure_compiled! in there should do the trick, too.

I really wouldn’t advise using ensure_compiled unless there is a really good reason to do so. OP is looking to create a compile-time dependency between two modules which is exactly what require and import do. Both will ensure their given modules are already compiled before the module they live in is. There are possibilities here where it might be needed but I’m not seeing that based on the provided code.

import/require would pull in functions/macros, too though, right? Curious what the best way would be to ensure a compile-time dependency without having that?

import yes, require no. require literally just ensures the module is compiled (namely that its macros have been expanded) before the module it’s called in, ie, required for the current module to work :slight_smile: It’s needed if there are macros in the dependency otherwise simply calling a function from it will suffice. For example:

defmodule Foo do
  Bar.a_regular_ol_function_that_is_called_in_the_module_body_for_some_reason()
end

This will make Bar a dependency of Foo. It doesn’t need a require if Bar isn’t calling a macro.

(sorry for all the edits if you noticed that, my dog is bugging me to go outside but wanted to finish the response, lol)

1 Like