Mix should recompile the project if you change any file inside “config”.
My guess is that you may be invoking a custom task that does not ask your project to compile. It is the responsibility of every Mix task to say which state they expect the application to be: compiled, loaded, etc.
If it still fails, can you push a sample project that reproduces the issue to GitHub? I will gladly take a look at it.