I’m working on a bullet-proof live reload mechanism in Hologram, and I need a reliable way to be notified when any module in a Phoenix application gets recompiled and its .beam file gets updated.
From my research, I’ve discovered that there may be a few ways to approach this:
Watching for .beam file changes - Set up a file system watcher to monitor changes to .beam files
What do you recommend? Is there another, more reliable approach that I’m missing?
I’m looking for a solution that’s as robust as possible, with minimal chances of missing any module recompilations, even in edge cases. Importantly, I want a mechanism that is not dependent on any Phoenix-related internals, just a pure Elixir way of doing this.
For my styled components library I used a Mix compiler to hook the compilation and then regenerate the compiled CSS from all of the modules. I didn’t do any checking of which modules were recompiled or anything, though. Figured I could do some diffing down the line if needed but I’ve yet to run into any performance problems. I think I stole most of the approach from Surface, which did roughly the same thing
Looking at those docs now I see they’ve added some new stuff (related to the LSP work, I believe). I’ll have to look into that!
I would recommend to take a look at source code how phoenix does it, as it has nothing phoenix-specific when it comes to code reloading (or at least it didn’t have when I was digging into it). The last time I took inspiration from there, I released this prototype library: BigBrother — BigBrother v0.1.0
PS: While that approach works rather well, there are inherent limitations from using the file watcher (which I believe phoenix no longer uses), for example if you are using a path library, the reload will not work.
I’ve actually implemented something similar in the previous iteration of the Hologram compiler, using a diffing mechanism to detect changes. However, to make this approach truly bullet-proof, I need to load all .beam files for all modules (including application code, dependencies, and the standard Elixir lib) and digest them for comparison each time, which isn’t very efficient. Ideally, I’m looking for a solution that can selectively detect just the modules that were recompiled or added/removed, rather than processing everything for diffing on each change. Thanks for the suggestion!
Thanks for sharing BigBrother! If I understand correctly, it watches for source code changes and helps reload modules, similar to Phoenix’s code reload, right?
What I’m actually looking for is slightly different - I need to detect when a module has already been reloaded, deleted, or added (and its .beam file was updated/added/removed). I don’t need to handle the reloading myself, just a reliable way to be notified after Elixir has done the work, so that I can re-transpile the changed modules and rebuild JS bundles.
I meant to link :beam_lib.cmp_dirs/2 directly. I guess the Discourse magic didn’t pull that into the preview.
In any case, cmp_dirs will tell you which beams have differing code between two dirs. For example you can compare a release dir on the system with a release dir in a fresh clone to decide which modules have changed. I have used this in the past to do targeted reloads of specific modules in running production systems, without a restart and without relup. (Of course you have to know a priori that the module you’re loading is safe, and does not change any assumptions about runtime state)
I realize it’s not exactly what you’re looking for but maybe a piece of the puzzle.
Oh, I got you now! I just assumed you were talking about :beam_lib as a whole. The cmp_dirs/2 approach could work for my use case, and I’ll need to benchmark to see how it performs. The old method just tracked module digests in a map, whereas this would require keeping the dirs contents (it’s only for dev env), but since changes are typically limited to a few modules at a time, it shouldn’t cause performance issues. I’d probably still have to pair this with a file system watcher to avoid polling.
Just to clarify something that seems to have caused some confusion - I’m actually looking to reload transpiled JavaScript, not the Elixir modules themselves. I just need to know when Elixir modules get reloaded so I can retranspile them and reload the relevant JS bundles.
