Module Does Not Recompile When Referenced Module Changes

mjlorenzo

Hi folks, got a question about how Elixir determines when to recompile a module. The documentation says a module will recompile when one of its dependencies changes, but that doesn’t seem to be the case here and I’m not sure why. Here’s the scenario:

I have a module that defines a schema for the library I’m building:

defmodule Schema do
  import Library.Schema.Notation

  schema :name do
    # macros and schema definition...
  end
end

When this compiles, it exposes a callback __schema__ to be consumed in another module. While Library.Client actually consumes the schema, Testbed is the intended product module:

defmodule Testbed do
  use Library.Client, schema: Schema
  #. (Library.Client.__using__() actually calls Schema.__schema__())
  # ... callbacks and implementation
end

I figured because Testbed references Schema, then it should know to recompile if Schema recompiles. I’m consistently seeing however that changes made to Schema do cause itself to recompile, but Testbed still doesn’t and I have to manually recompile it to reflect the changes in Schema. This ring a bell for anyone?

Thanks,
Mike

14 comments

#compiling

1 977 14

2024-03-06 16:04:13 UTC

Showing Posts 1 to 10

Show Best Posts
Show All (oldest first)
Show All (newest first)

sodapopcan

With an oversimplified mix project, I am unable to reproduce this:

defmodule Recompile.Schema do
  defmacro __schema__() do
    quote do
      "I do schema stuff!"
    end
  end
end

defmodule Recompile.Notation do
  defmacro __using__([schema: schema]) do
    quote do
      require unquote(schema)
      unquote(schema).__schema__()
    end
  end
end

defmodule Recompile.TestBed do
  use Recompile.Notation, schema: Recompile.Schema
end

$ mix xref graph --label compile --sink lib/recompile/schema.ex
lib/recompile/test_bed.ex
└── lib/recompile/schema.ex (compile)

and when I change Schema:

$ mix compile --verbose
Compiling 2 files (.ex)
Compiled lib/recompile/schema.ex
Compiled lib/recompile/test_bed.ex

Could you possibly share more code and your Elixir version? Is this in a mix project?

Post #1

awerment

I’m able to reproduce it with a __schema__() function (not macro) and without the require unquote(schema).

Think the issue might be with the fact that schema in unquote(schema).__schema__() is dynamic and there is no hard dependency on the module, in might be loaded at runtime after all. Adding Code.ensure_compiled!(unquote(schema)) inside the quote block in defmacro __using__(..) works for me.

defmodule Schema do
  def __schema__(), do: :ok
end

defmodule Client do
  defmacro __using__(schema: schema) do
    quote do
      Code.ensure_compiled!(unquote(schema))
      # ...
    end
  end
end

defmodule Testbed do
  use Client, schema: Schema
end

Post #2

sodapopcan

My first attempt was with a non-macro __schema__() and no require and still couldn’t reproduce. I just changed to a macro to more closely fit what I imagine the real code is.

Post #3

sodapopcan

Ah ya, in your example simply doing unquote(schema) isn’t going to do anything anything meaningful if you’re just using a regular function in the way you have it. You’d need to do import unquote(schema) for __schema__ to be available in Testbed.

Post #4

awerment

Yeah, I’m guessing the call to __schema__() is itself inside a function definition within the macro’s quote block? If it’s called outside as in your example, it works for me, too.

Post #5

sodapopcan

It’s just that saying unquote(schema) on its own is going to expand to just Schema which is simply the atom Schema which is a no op. If you do import unquote(schema) that will actually import its functions. You don’t need ensure_compiled there as import ensures its compilation.

Post #6

awerment

That’s true, I assumed @mjlorenzo’s Library.Client module would look something like this:

defmodule Client do
  defmacro __using__(schema: schema) do
    quote do
      # Code.ensure_compiled!(unquote(schema))
      def my_schema(), do: unquote(schema).__schema__()
    end
  end
end

In that case, without Code.ensure_compiled!, Testbed is not recompiled when Schema changes.

Post #7

sodapopcan

Yes, that is correct behaviour in this situation because it’s inside a function which makes it a runtime dependency.

Post #8

mjlorenzo

@sodapopcan, @awerment thanks y’all for the discussion. This is the top of Library.Client.__using__:

 defmacro __using__(opts) do
   raise_if_no_schema(opts)

   schema = opts[:schema]
   module = Macro.expand(schema, __ENV__)
   Code.ensure_compiled!(module)
   defs = module.__schema__()

If i’m understanding correctly, because the above is entirely in the macro body of Library.Client.__using__, Testbed itself has no idea to interpret Schema as anything other than the atom and so no dependency exists. Injecting some sort of import, require, etc into Testbed will establish this. On the right track?

Post #9

sodapopcan

Mostly, yes. Nothing outside of the returned quoted expression will be injected into Testbed so once that module is compiled, all that stuff is gone. So in this case it’s not that it isn’t interpreting Schema as anything other than the atom it’s that Schema isn’t even present in the compiled code! Again it’s hard to see without more context but there doesn’t seem to be anything about that code snippet that suggests you need Macro.expand over requireing or importing within a quote block which of course you want anyway to create the dependency. It depends what you’re trying to accomplish, of course.

I’ve also never actually seen ensure_compiled! used inside a module before. I’m not a master of macros, though, so maybe there is a good reason, but AFAIK, ensure_compiled! is essentially require for use outside of a defmodule. Don’t quote me on that, though