Avoiding re-compiling via build_per_environment: false

I had a question about using build_per_environment: false in my mix.exs… I have a simple app that doesn’t have any compile-time configuration, so I thought I could use build_per_environment: false so I wouldn’t have to compile the app twice (once for dev and once for test). Including build_per_environment: false in my mix.exs does cause the .beam files to get compiled into _build/shared/lib, but the app still has to compile once per environment, which is not what I expected. The app compiles when I start it in dev via iex -S mix and it re-compiles when I run tests via mix test. Why is it re-compiling? Am I misunderstanding what build_per_environment: false does? Or how the compilation works? Thanks for any clarifications!