As an aside, this surprised me too. I quickly realized ‘how’ it works, but it still seems super weird to me to this day and I still mentally try to parse it as a run-time configuration instead of a build-time configuration…
OvermindDL1
josevalim
Creator of Elixir
I really don’t think that the choice of my Ecto adapter or my logger compile_purge_level belong in mix.exs. The mix.exs is about how Mix compiles my project and Mix as a build tool in general, it is not about how my application interfaces with something as important as a logger.
I also don’t believe that users should have to know or care if a configuration is compile time or runtime unless they really have to. Imagine how much confusion we will have if now my Logger configuration is spread out in two places. The only time not knowing if a configuration happens at runtime or compile-time will be an issue is if the user expects something to happen at runtime but that is a compile time only option. That’s exactly where on_boot can be helpful: if we allow the user to declare their expectations via something like on_boot, we can detect those cases programatically.
No, you can’t configure logger in your own def application do.
This issue is not about on_boot at all. Supporting releases is what will make the distinction between compile-time and runtime exist. The current proposal, as originally written, has the exact same compile vs. runtime issue as on_boot, except it is implicit and left up for the users to guess. on_boot is about making those differences explicit.
I will be straight-forward here: this is out of discussion. config.exs is executed at compilation time. It has been behaving like this for at least 4 years. It doesn’t matter how people think it behaves. Compile-time is how it behaves right now and we are not trying to change it. And libraries like Nerves, Phoenix, Ecto and Logger rely on this behaviour at different places. Suggesting to find a new home for all of those common configuration patterns would require a long-term migration effort (as both libraries and config files need to be rewritten). It is impractical at best (and IMO just plain undesirable). While I would agree some of this compile-time configuration should actually be moved to lib (such as the Ecto adapter), those few cases are rather the exception.
In fact, all of the alternatives brought up so far, such as command line application configuration, is not related to compile-time configuration, but rather to runtime configuration. In a release, the only thing I can specify at the command line, by definition, is runtime configuration. So if we really don’t want to mix compile time vs runtime, then the only option is to keep the config.exs as is and outside of releases, and use tools like command line options, Phoenix’ init/2 callback and friends for the runtime configuration.
3
talentdeficit
Rebar3 Core Team
In fact, all of the alternatives brought up so far, such as command line application configuration, is not related to compile-time configuration, but rather to runtime configuration. In a release, the only thing I can specify at the command line, by definition, is runtime configuration. So if we really don’t want to mix compile time vs runtime, then the only option is to keep the config.exs as is and outside of releases, and use tools like command line options, Phoenix’ init/2 callback and friends for the runtime configuration.
i like this proposal best of all. retain config.exs for compile time only and let it continue to work as it does. move runtime config to some other place
i think the confusion users experience in the difference between compile time and runtime is entirely a result of encouraging the use of the same file/process for both. separating them is probably the best thing that can be done to resolve this
3
josevalim
Creator of Elixir
The import_config change is necessary if we want to execute the same files in a release as we do for Mix. So we need to track the files. If we have on_boot, we only need to track the code inside on_boot.
The reason the confusion exists is because the compile/runtime is completely irrelevant for Mix but it exists in a release. Doing nothing means the major mismatch we see between Mix and releases in regards to config won’t be addressed.
Unless we strictly impose Mix to be a compile-time config only. This means that MIX_ENV=prod PORT=4040 mix run won’t have any effect if the project was already compiled with PORT=8080. But going down this path feels like we are breaking both approaches, instead of fixing them.
0
OvermindDL1
Eh, only in so far as mix is ‘running’ like ELIXIR_BLAH='something' mix compile or mix compile -DELIXIR_BLAH=something or so.
Actually the definition is intensely important in normal code as well. Like the URL for the ueberauth_cas library gets hardcoded into the compiled code at compile time from the config, when it by all accounts should be runtime changeable (and I had to have it to that, hence forked). Yes this is a library issue, but the current style ‘encourages’ this usage, which is a big problem.
Except an environment variable should take precedence over a higher staged default value, thus 4040 should be used over the 8080 unless the code does something highly irritating like bake it into the compiled beam (which, again, is what the current config system encourages people to do)…
2
yordisprieto
I am not a big fan of adding any complexity to the mix.exs file I would prefer to go with the on_boot macro and keep all the config related code in the config files.
I am kind of worry about library creators adding the configuration to their mix.exs file.
How that would work then?
Right now it is simple and I dont worry about config files in my deps because they are not being used.
Are you gonna ignore that config as well?
That is my main concern when you push anything to the mix.exs file instead of living it in the config files.
But again going back to the first sentences, no mix.exs has metadata that shouldn’t be important for must of the library packages and I would love to keep it that way.
And is better to rely on copy-paste configs than library authors handling all possible use cases for the configs this is my biggest concern.
Definitely what I expect from configuration perspective, but
config_path or config_files because it seems that every config is actually a file, no a folder so that confuse me from language perspective. Sorry for the dumb question.
Just a comment: Only for production release, I wouldn’t like to have my test configs per app in the root of the umbrella app (it becomes too large with small benefits)
0
yordisprieto
Standalone preferences so far,
Add macros to the config files like on_boot.
DevOps and engineers have one place to deal with configuration issues and library authors do not have to deal with configurations for you specific use case (unless you ignore every config from the mix.exs proposed so far)
0
josevalim
Creator of Elixir
Exactly. The only time a user cares about runtime/compile is when they want to configure something compile-time at runtime. on_boot would make those cases clearer, hence this discussion. If in the future we add metadata to applications that allow them to tell which configuration happens at compile-time, we would even be able to catch those errors programatically early on.
In any case, it is likely that the configuration you mentioned shouldn’t even be in the application environment at all, regardless if it is compile-time or runtime. Because even if you move something to be a runtime configuration, it is still a global setting, that we want to avoid nonetheless. Hence the latest recommendations in the library guidelines.
At the end of the day, the fact there are some illegitimate cases of compile-time configuration does not imply that they aren’t legitimate cases for compile-time configuration. Those are quite common and we are not breaking them. If we want to make it harder to leverage compile-time configuration, then we need to make it harder for library authors, not users. We shouldn’t “punish” users because of mistakes library authors make.
5
cybrox
You’re definitely right about that, I agree that it is not used consistently enough. The point I was trying to make is, that it is an already existing, elegant and simple possibility of solving a large part of the issue at hand. - Albeit not as clean as solutions proposed here.
I assume this is mostly because a lot of people try to stick to paradigms set by the “big players” in terms of frameworks / libraries and follow the phoenix style of splitting configs per-env, which allows for hard-coding those parameters for development environments. Personally, I agree with you, I use environment variables for configuration in development as well.
This is actually a very interesting idea, imo. It would allow for extending the current possibilities in a much more dynamic way without adding bloat and complication to the mix.exs file.
This is probably the most important point of what sparked the need for a proposal like this. I can’t help but think that introducing a parallel mechanism for doing things (Application.config) will add more complexity than required. The current set of tools would already suffice for solving this issue, if only we would come to an agreement on how to split those entirely different parts of configuration.
I would like to disagree, depending on how libraries are implemented, this difference is crucial to ones understanding of their own configuration. I that regard, I personally think the current way of config option x is compile-time, unless explicitly implemented otherwise is very clear. The big issue is, that most people are not aware how - and a lot of libraries don’t allow - to even use runtime configuration.
Yep. As someone who has working tooling for my applications in place, a change like this would be terrible. We are trying to solve an issue that is not ideal, it’s not something that’s broken, thus adding possibilities instead of just replacing working solutions would definitely be the way to go.
Exactly. I think @OvermindDL1’s proposal for dynamically importing one other configuration file would be an awesome solution for adding this possibility without introducing unnecessary complexity.
- The user can still use
config/{config|env}.exsfor compile-time config - The user can still use
System.get_env/1etc. there, not breaking existing configs - The user is more aware of the static nature of those configurations
- The user can explicitly pick a config without additional code
tl;dr: I really like RUNTIME_CONFIG=config/myconfig.exs bin/myapp start
2
sasajuric
Author of Elixir In Action
Personally I believe that a lot of the stuff which is managed by config.exs & friends belongs to the code. For example these are params which mix phx.new by default pushes into config script, spread across four files, bundled with unrelated data, but detached from the place where it logically belongs.
Why should by default this data reside in config scripts, and why at runtime must it sit in a public global mutable storage in a separate memory space?
No, but we could guide users to use approaches which make sense. I think that a lot of misuse of config scripts originates from the fact that generators promote them by default.
5