For a while we used things like deferred_config to pull env vars into config at runtime. This worked, but there were setting that we’d like to update while the app was running. We did this by writing a library that uses configparser_ex to parse ini files and file_system to monitor a config directory for changes to ini files. When the library app starts it loads any ini files found and stores the data in an erts table. We have a github checkin hook that will update ini files in production when changes are checked in. The library gets a notification and reloads the ini file that changed and updates erts. This allows us to update config while the application is running.
We treat config/config.exs as compile time settings that don’t change often (or ever.) And all other settings go into the ini files.
I would love to not have to care about it too, so we agree there. But I don’t know that it’s avoidable. Either way, I guess I would rather force us to be explicit and have things work in an understandable way as opposed to things being implicit and wrong in a lot of cases. on_boot is still an explicit mechanism, just one that doesn’t seem obvious to me at first glance.
I guess that is true. If there’s a way for me to look up an item and say “yes this config option can be modified at runtime”, therefore I need to move it somewhere else… I guess, to me, on_boot (or whatever it ends up being called) or a different file doesn’t make a huge difference at that point. I think that passes my test for being explicit. Be default, everything is compile time but some things are made special and become runtime (if applicable).
I’ve read over this thread, and I have some thoughts about the situation. Why can’t we just keep config files as static, compile-time only configuration, and use the code for runtime config? It seems ideal to me since callbacks like init seem to serve this very purpose. That would make it very explicit as to what is configured at runtime vs. compile time, since you are configuring these things in two different places, as they should be, since runtime config should be part of the code IMO.
It would make a lot more sense and would be more explicit IMO to use System.get_env in your code than in a config file.
This is the most recent thing we tried in Phoenix and Ecto via the init/1 callback and it caused a lot of confusion because configuration is now broken in two places. So someone would try to set a static flag in their config.exs and nothing would happen. They would spend some time debugging the problem and it turned out that the flag was being overridden in the code with an environment variable.
Although one is runtime and the other is compile-time, they are still configuration, and having them in two completely different places can lead to the confusion I above.
It is not much different from having separate files for compile and runtime, which was proposed earlier. It is a valid approach and with more flexibility, but it will likely lead to file explosion too.
Indeed, it is similar, but it does solve two issues that the separate-files proposal had:
order of file inclusion is more clear because this is explicit in the list of paths.
All of these files are optional so there is less of a file explosion: Creating multiple config files is only necessary once you have configuration that should be specialized to a certain configuration target.
I have a politics related question: how willing is Elixir’s core team to introduce a breaking change that forces everybody to migrate to a seemingly perfect solution (which is yet to be discovered)? This should only come with the next major version however – say, Elixir 1.7.
My experience with people says that they will not change their configuration files for years and will only do so if a version upgrade mandates it. Due to that, it’s my opinion that when the solution is found it definitely should be made very explicit and impossible to ignore. Otherwise a lot of library authors will keep coming up with creatively flawed solutions forever.
What’s your opinion on this political matter?
EDIT: Obviously, a transitional period with deprecation warnings should be around for a while before a breaking change is introduced.
I’ve had such issues myself with just config scripts. In particular I went down a couple of wild goose chases because I didn’t notice an override in another .exs script, or there was an override in the code (either as a compile-time if on Mix.env, or as a runtime if on some other non-obvious global).
Therefore, the problem is IMO not in init/2 but in the fact that parameters for endpoint are dispersed across multiple places (which they already were even before init/2 was introduced). If on top of it people used one more place for providing parameters, I’m not surprised they were even more confused.
I’d say though that this wasn’t a proper evaluation of init/2. I’d wonder if the people would face the same problems if they provided the complete parameters to the endpoint in a single place (init/2 callback).
If the guidelines for library authors say we should not use application configuration unless strictly necessary, then we should make it less-convenient for them to do that, so that we have less problems with configuration being in multiple places. It feels like the proposal being discussed in this thread makes it easier for both library users and library authors to continue to not follow that advice.
Additionally, introducing a dynamic step that happens before application boot to run your configuration script will introduce a new failure mode that would presumably be outside of the normal supervision hierarchy. This leaves the developer with less opportunity to deal with failures and less opportunity to bring up parts of the system that they want to be online even if the configuration is invalid or some “side-effect” of the configuration script fails on first run, using the normal OTP toolbox that we have today.
I’m not saying make it harder to ‘define’ configurations for a given stage, but rather it should be made harder to ‘use’ a configuration from the wrong stage. The onus should be on the library author to do it right, not the library user.
This this this right here!
Exactly! Operator configurations belong in the system environment, not internal to a baked release application or so (thus system environment, external json, JVM’ish XML, whatever…).
I would say yes, or rather have a runtime configuration delegate to the build-time configuration if not otherwise specified (thus giving a nice fallback order).
An issue here though is that the config scripts dump into the application environment regardless, mixing up build time and run time configurations, which is why it is so easy to mis-use one that should, for example, be a run-time config at compile-time (like say a port number), this is because the interface to access both are identical when their stages are not the same.
Yes, definitely this, the interface needs to be different to get each stage’s configurations.
And honestly even run-time should not be able to access compile-time’s configurations either (just like how mix is not available at run-time, or not supposed to be anyway).
I actually edit this file fairly often to change boot-up configs… >.>
Same here, and the fact that almost no library responds to change made here is maddening, I have to kill entire process trees to get them to update (if they even update, which they won’t unless they do stupid things like baking it into the source by accessing the config at compile-time for something that should be run-time) or I have to restart the whole VM (which is when I just edit sys.config then…).
I do have complaints here, even something as simple as my discord bot elixir server has hard-coded configs that I have to change at times because of mis-designs in my dependencies that I have to deal with…
And yet you ‘can’ do things like change the purge level configuration and people have been surprised that it doesn’t work until they recompile.
