I’m trying to track down why my testing experience is such a pain because there’s always many files being compiled… I’ve used the inotify trick from Understanding Elixir's recompilation · Milhouse to see which files these are (is there a simpler way?) and tried to understand from mix xref why those files were compiled.
So there’s runtime, compile time and exports dependencies, I learned from mix help xref. So when I do a trivial change like adding a newline in a source file, only compile time dependencies should trigger a recompile.
If I change file A, all files having a (transitive) compile time dependency on A would need to be recompiled.
This lead me to use mix xref graph --label compile --only-direct because my reasoning is that this would show me all non-transitive compile time dependencies, and I should be able to traverse from the changed file to every compiled file via a graph node connection in the output of this command.
Here’s a snippet from the output, the email.ex file has only one dependency shown.
The email file was recompiled when I made a trivial change in a file which, according to mix xref (with the invocation above), was not connected at all to the email file.
If you are using Bamboo (guessing based off the email view), it was a pretty bad offender in that regard. Luckily the latest release improves the situation significantly:
Now all of your email-related modules won’t recompile all the time (or maybe it’s vice versa).
There’s a pattern when writing macros to write as little code as possible inside the macro itself, but use functions as much as possible. The simplest such scenario is calling include to import functions into the calling module. This idea has lead to code like this:
defmodule Fancy do
defmacro __using__(_) do
quote do
# do some stuff...
import unquote(__MODULE__)
end
end
def utility_function do
# calls other module
M.other_fun()
end
end
When A uses Fancy, we have a compile time dependency A => Fancy
Due to the function inside the Fancy module, there is also a runtime dependency Fancy -> M
So when M changes, A needs to be recompiled.
In my case, A had a transitive runtime dep to the Phoenix router (for calling helpers), which created a huuuuuge recompilation loop.
The easiest solution here is to break the utility functions in Fancy out into a separate module (and file).
Thanks for the advice, I quickly found the culprit:
I had compile time dependencies from the router to two controllers. Because those controllers had a transitive run time dependency back to the router (redirects using route helpers) and from there to every other controller, changing any controller required the router to be recompiled.
The fix:
# change this:
get "/", MyApp.FooController, :index
# to this:
scope alias: MyApp do
get "/", FooController, :show
end
simply because now the fully qualified module name of the controller it no longer present in the router.
Two other things I did/tried:
elixirc_options: [verbose: true]
in the project/0 callback of the MixFile, which made the effective dependency chain more obvious.
I also tried setting
config :phoenix, :plug_init_mode, :runtime
not only in dev mode, but in test mode as well, assuming that plug initialization would otherwise happen at compile time and that that would also be the case for the controllers being routed to (since they are plugs as well), but I was wrong here, the default :compile does not make all controllers compile time dependencies of the router.
I got tired of analyzing the output of mix xref so I wrote a thing… If you run into this problem please check out GitHub - schnittchen/why_did_recompile and help me improve it
…And I tried it. Does what it promises. My problem is that the project I am participating in is huge (1100+ Elixir files) and the output isn’t giving me a direct feedback what to address because the transitive dependency chains are like 40+ nodes.
I used verbose compilation and just touch-ing files and I despaired.
Wish Elixir had compiler apparatus to tell you which exact statements in the source file lead to compile / runtime dependency between files. Some of the files are huge, there are use statements, there are macros, there are Phoenix peculiarities (dependencies between router and views and various utilities) so you can very easily get lost in all this.
Not your fault, of course, the tool is useful! Thank you for it.
This sounds oddly familiar to an issue I experienced on an Erlang project. The project was passing macros to erlc at compiletime with -d, and due to a dependency chain it caused a whole-app recompile for each change. Could something similar be happening with your Elixir project? This escaped xref and other tools completely.
That is not quite accurate, an alias will only cause a run-time dependency. However, compile-time dependencies are transitive so if A->compile-time->B and B->run-time->C, then when C changes, A will need to be recompiled.
Did you have any success in finding a strategy for diagnosing your issue?
I’m in a similar situation, working on a large project with a ton of files. In some cases changing the css class in a template results in hundreds of files being recompiled. Turning on verbose compilation is somewhat helpful but as you say trying to track things down gets overwhelming quickly!
Nope, unfortunately gave up. The company I worked for back then was extremely output-driven so doing R&D was frowned upon unless you achieve results in a day or two which is unrealistic for such a hard and exploratory task. We parted ways soon after.
Since then I haven’t worked with bigger code bases in Elixir – only in Rust. Those I work with in Elixir are small to medium.
I’d love to work on this again one day – there’s a potential to improve things by piping several CLI tools at the tail of mix xref ... – but between my constant fight to undo a potential serious health condition and working part-time, it seems it ain’t happening for now.
Some of the files are huge, there are 
I think it could be very useful for smaller projects.
