MrYawe

MrYawe

Recompile_buster - find and fix the files that cause the most recompilation

Hey everyone!

I just released recompile_buster, a Mix task that analyzes your project’s xref graph to surface the modules whose changes trigger the most transitive recompilation.

If you’ve ever changed one file and watched half your project recompile, this tool helps you figure out why and where to fix it.

I was working on a large Elixir codebase where incremental compilation times kept growing. mix xref graph gives you the raw data, but it’s hard to know where to start when you have hundreds of files. I wanted a tool that would rank the worst offenders and give actionable next steps.

It’s also useful as a CI guard, the --fail-above flag lets you fail the build if the number of problematic files exceeds a threshold, so you can prevent compile-time coupling from silently creeping back in.

Here is an example using it in the plausible analytics codebase:

> mix recompile_buster

Files causing excessive recompilation (sorted by transitive deps, 3 levels):

  # | Trans. deps | Recompiles | File
----+-------------+------------+---------------------------------------------
  1 |         308 |          1 | lib/plausible/prom_ex.ex
  2 |         130 |          1 | lib/plausible_web/tracker.ex
  3 |         123 |          2 | lib/plausible/imported.ex
  4 |         114 |          1 | lib/plausible/auth/api_key.ex
  5 |         112 |          2 | lib/plausible/billing/plan.ex
  6 |         109 |          1 | lib/plausible/ingestion/event.ex
  7 |          95 |          1 | lib/plausible/props.ex
  8 |          93 |         32 | lib/plausible_web/live/auth_context.ex
  9 |          90 |          1 | lib/plausible/billing/plans.ex
 10 |          88 |          2 | lib/plausible/billing/feature.ex
 11 |          88 |          7 | lib/plausible/imported/google_analytics4.ex
 12 |          83 |          1 | lib/plausible/stats/imported/base.ex
 13 |          77 |         11 | lib/plausible/imported/importer.ex
 14 |          71 |          1 | extra/lib/plausible/site/tracker_script_id_cache.ex
 15 |          65 |          1 | lib/plausible/stats/sql/expression.ex
 16 |          63 |          7 | lib/plausible/imported/csv_importer.ex
 17 |          55 |          5 | lib/plausible/imported/site_import.ex
 18 |          52 |          4 | extra/lib/plausible/stats/goal/revenue.ex
 19 |          50 |          2 | lib/plausible/sites/index.ex
 20 |          49 |          7 | lib/plausible/imported/universal_analytics.ex

Problematic files: 36 | Total transitive deps: 2357

To start investigating the most problematic file, run:
  mix recompile_buster --explain lib/plausible/prom_ex.ex
> mix recompile_buster --explain lib/plausible/imported/csv_importer.ex

Explain: lib/plausible/imported/csv_importer.ex
===================================
Unique transitive deps (3 levels): 63 | Direct deps: 7 | Recompiles: 7

A) Files that recompile when csv_importer.ex changes (7):
  lib/plausible/imported.ex
  lib/plausible/imported/import_sources.ex
  lib/plausible/imported/site_import.ex
  lib/plausible/stats/imported/base.ex
  lib/plausible/stats/imported/imported.ex
  lib/plausible_web/live/imports_exports_settings.ex
  lib/workers/local_import_analytics_cleaner.ex

B) Direct dependencies of csv_importer.ex (63 unique transitive deps):
  lib/plausible/imported/site_import.ex (27 transitive deps)
  lib/plausible/imported/importer.ex (24 transitive deps)
  lib/plausible/s3.ex (14 transitive deps)
  lib/workers/local_import_analytics_cleaner.ex (7 transitive deps)
  lib/plausible/clickhouse_repo.ex (4 transitive deps)
  lib/plausible/repo.ex (4 transitive deps)
  lib/plausible/ingest_repo.ex

Every file in (A) will recompile if any of the 63 files in (B) changes.
To fix this, break the compile dependency (A) or reduce the transitive deps (B) by removing direct dependencies with the most nested deps.

To investigate why a specific file is in (A), run:
  mix xref graph --source <file> --sink lib/plausible/imported/csv_importer.ex --label compile

https://github.com/MrYawe/recompile_buster

Most Liked

michallepicki

michallepicki

The most common problem is: when a module A depends “at compile time” on module B, and module B depends (in any way, even just at run-time) on module C, then A will recompile when C changes. So it’s good practice to move macros, constants etc to their own separated modules that don’t depend on anything else, to avoid unnecessary recompilations

ream88

ream88

This should be part of every Elixir app: From 7 problematic files / 386 transitive deps to zero. in 5 minutes using Claude Code! Incredible work!

Where Next?

Popular in Announcing Top

sasajuric
I’d like to announce a small library called boundaries. This is an experimental project which explores the idea of enforcing boundaries ...
New
ostinelli
Let’s write a database! Well not really, but I think it’s a little sad that there doesn’t seem to be a simple in-memory distributed KV da...
New
josevalim
Yes, yet another parser combinator library! Most of the parser combinators in the ecosystem are either compile-time, often using AST tra...
159 19483 141
New
Crowdhailer
Experimenting with this code. OK.try do user &lt;- fetch_user(1) cart &lt;- fetch_cart(1) order = checkout(cart, user) save_orde...
New
RobertDober
Earmark is a pure-Elixir Markdown converter. It is intended to be used as a library (just call Earmark.as_html), but can also be used as...
239 12673 134
New
sbs
Only 650 LOC, wrote for fun :slight_smile: https://github.com/sunboshan/qrcode
New
woylie
I released Doggo, a collection of unstyled Phoenix components. https://github.com/woylie/doggo Features Unstyled Phoenix components....
New
markmark206
simple_feature_flags is a tiny package that lets you turn features on or off based on which environment (e.g. localhost, staging, product...
New
OvermindDL1
Been making an MLElixir thing (not released yet…) for fun in spare time in the past day. I’m just trying to see how much I can get an ML...
132 14085 106
New
New

Other popular topics Top

sen
Hi All, I set a environment variables in dev.exs , like below code. when i start server, how can i set the ${enable} value? thanks. d...
New
lastday4you
I wanted to check elixir version in phoenix because i found that my elixir is 1.5 but when i use Enum.chunk_by it said the function is un...
New
mcarvalho
What is the difference between System.get_env and Application.get_env? For example, what are best practices to use one versus another.
New
skosch
To my knowledge, put_in, Map.update etc. all have the one limitation of not automatically creating intermediate keys when needed (for exa...
New
lessless
I believe there are people here who are dealing with CSV files import on the daily basis, and since Excel is a really popular tool there ...
New
Patoshizzle
After calling mix ecto.create I get this error: 17:00:32.162 [error] GenServer #PID&lt;0.412.0&gt; terminating ** (Postgrex.Error) FATAL...
New
JeremM34
Hello, how can I check the Phoenix version ? Thanks !
New
jason.o
In the code below, if the create action is not set to accept “extra_key” as an input, it errors out with a message shown above. Is there ...
New
boundedvariable
I am going through the kafka architecture. All the features what the kafka is providing are already in Erlang. I would like hear your opi...
New
rms.mrcs
Hi, I need to transform a list of numbers into a map where the keys are the indexes and the values are the original values of the list. ...
New

We're in Beta

About us Mission Statement