Excessive recompilations when nothing substantially changed

If A depends on B at compiletime, then any transitive runtime dependency of B is a compiletime dependency of A. As As compiletime is Bs runtime.

If you are using Bamboo (guessing based off the email view), it was a pretty bad offender in that regard. Luckily the latest release improves the situation significantly:
https://github.com/thoughtbot/bamboo/pull/559

Now all of your email-related modules won’t recompile all the time (or maybe it’s vice versa).

Thanks, that helps a bit.

Thank you, that makes sense, I had not thought of that.

There’s a pattern when writing macros to write as little code as possible inside the macro itself, but use functions as much as possible. The simplest such scenario is calling include to import functions into the calling module. This idea has lead to code like this:

defmodule Fancy do
  defmacro __using__(_) do
    quote do
      # do some stuff...
      import unquote(__MODULE__)
    end
  end

  def utility_function do
    # calls other module
    M.other_fun()
  end
end

When A uses Fancy, we have a compile time dependency A => Fancy
Due to the function inside the Fancy module, there is also a runtime dependency Fancy -> M

So when M changes, A needs to be recompiled.

In my case, A had a transitive runtime dep to the Phoenix router (for calling helpers), which created a huuuuuge recompilation loop.

The easiest solution here is to break the utility functions in Fancy out into a separate module (and file).

I still have the problem that when M is changed, my router gets recompiled, but the only relevant dependencies seem to be these runtime dependencies:

Router -> Controller -> M -> Router

I know a cycle might be bad, but since there is no compile time dependency involved I don’t get why the Router is recompiled.

Is the project public, or can you reproduce it in a fresh project that you can make public?

I’m afraid it’s not public. I think trying to reproduce it from bottom up is the right approach. Don’t know when I can get to that…

Thanks for the advice, I quickly found the culprit:

I had compile time dependencies from the router to two controllers. Because those controllers had a transitive run time dependency back to the router (redirects using route helpers) and from there to every other controller, changing any controller required the router to be recompiled.

The fix:

# change this:
get "/", MyApp.FooController, :index
# to this:
scope alias: MyApp do
  get "/", FooController, :show
end

simply because now the fully qualified module name of the controller it no longer present in the router.

Two other things I did/tried:

elixirc_options: [verbose: true]

in the project/0 callback of the MixFile, which made the effective dependency chain more obvious.

I also tried setting

config :phoenix, :plug_init_mode, :runtime

not only in dev mode, but in test mode as well, assuming that plug initialization would otherwise happen at compile time and that that would also be the case for the controllers being routed to (since they are plugs as well), but I was wrong here, the default :compile does not make all controllers compile time dependencies of the router.

I got tired of analyzing the output of mix xref so I wrote a thing… If you run into this problem please check out GitHub - schnittchen/why_did_recompile · GitHub and help me improve it