If your module A only depends on the struct of a module B, then we will only recompile A if the fields in the struct B change. But we always recompile the whole file the module is at. The beam of a module is always assembled as a unit.
A lot of it still comes from things like plugs and phoenix helpers, which are effectively fixed in the next Phoenix v1.4 version.