Runtime configuration library (with casting, validation etc...) for native releases

That’s it, the way to go when dealing with environment values for production.

Defaults values may lead to have production running with credentials of type user: admin and password: admin , just to give the most obvious example.

Yeah, we definitely don’t want to set these to default in prod (and we don’t!). However, there are occasional examples where a prod default might make sense. For example, we want to allow connection pool to be configurable, b/c it might help an operator patch a production which is falling apart, but we still want to provide some “sensible” default. Another example could be logger level (use info by default, make it possible to reconfigure without needing to redeploy the system). But yeah, I agree that in most cases defaults are best avoided (e.g. db name, credentials, api tokens & such).

Vapor looks really great but then how do you deal with differing config based on “mode”?

Is there an example “hello phoenix” repo out there using Vapor configuring Ecto and Phoenix for dev, test, and prod? I did a github public code search for uses of :vapor in the wild and couldn’t easily find examples.

Thank you!
The solutions you mention definitely make sense, and would be what I would do first as well as long as I am the owner of both parts of the code. They break down if we are writing an application that uses a library, so we have two authors, each in charge of different parts of the code.
If you want to use the library twice, or configure it differently during testing, but the library owner has only specified a way to do ‘global configuration’, then you, the application writer, are out of luck.

Some examples would be a for instance a couple of libraries I wrote earlier that expose data structures like functional stacks/queues/vectors/priority queues. Which one of the concrete implementations to use for these abstract datatypes depends on whether you want to optimize for read efficiency, write efficiency, memory efficiency or e.g. require real-time efficiency over an amortized one. Those decisions are usually things we want to leave for the application user, even if we use the datatypes in an intermediate library, because in the end we decide at the application level whether we run the code on an embedded device or a giant VPS, which is where these trade-offs start to matter.

I actually think that what @keathley means is influenced by the fact that when you configure certain tools/frameworks/libraries your application is using, these configurations are actually static between environments. Furthermore, only hostnames and passwords are security-sensitive, things like e.g. pool sizes are not.
I expect that their setup uses a .env for the non-sensitive settings, and another set of ‘real’ environment variables that contains the sensitive fields amends these in production, with the .env.local file providing a variant of those sensitive settings (and other overrides) for development.

This is exactly what I mentioned to be a bad security practice. It should be inverted.

A .env file should never be committed into your version control, because it’s normally a place where you set sensitive data. Just take a look into all dotenv libraries across many programming languages, and you will see this pattern off using .env as a private file, aka not shared with others through git repos.

Please be open to the ideas that:

  • Most people aren’t cavalier with security concerns when made aware of them, or especially if they were already aware of them before your remarks
  • Other people’s experiences and values may be different than yours and that’s okay

Combining the two leads us to the reminder that you probably don’t need to scold a thread full of other users for a practice they’ve probably given a great deal of thought to individually as well as received feedback from other peers within their organizations about. They probably didn’t enact the technique unilaterally and against internal objections.

2 Likes

And the users, like newbies, that come here and take this examples as a good practice?

You haven’t established that it is in fact a bad practice - “it’s subjective, and possible to be abused, but not automatically wrong in all cases generally” is what I’m trying to get you to reflect on and internalize. If you want to make a principled argument about how it’s dangerous based on reasons XYZ, fork a new forum thread about it and try to convince people. You’ll probably get a lively discussion. Simply stating that it’s insecure is an unsubstantiated claim thus far.

Typically in such cases the lib will require app env, and if I want to test this, I’d do it by changing the app env from an async: false test.

I see no relation between that and the design of our provider abstraction. Our abstraction fetches values from external storage, but it doesn’t store them anywhere. In practice, we do use some of these values to supply a few of libs which require app env, which we do by setting the app env in the app start callback before the supervision tree is started.

I have to say that you lost me there :slight_smile: . These are typically developer’s concerns, not end-product operator’s. Our abstraction is all about operator config, i.e. it is a mechanism which allows the person operating the system to configure it. I have trouble imagining a scenario where such a person would have to choose between one stack implementation or the other, but even if such scenario did exist, I’d take the configuration as a simple choice (i.e. the operator would either supply “foo”, “bar”, or “baz” for the stack option and the program would use that to pick between StackFoo, StackBar, or StackBaz).

FYI, our library is now open. You can find the code here. There’s a basic tutorial on the repo readme, and some further details in moduledoc. Any feedback is very welcome :slight_smile:

8 Likes

Elixir 1.11 is coming with config/runtime.exs and mix app.config.

https://hexdocs.pm/mix/1.11.0-rc.0/Mix.Tasks.App.Config.html

Loads and configures all registered apps.

This is done by loading config/runtime.exs if one exists.

It means… if we move config validation from when the app starts to config/runtime.exs … we can use this task to check the config issue upfront.

For example, I usually have a helper function on supervisor to read config and pass it to child specs. That is only executed when the supervisor starts (so requires app start).


Also… I recently found GitHub - dashbitco/nimble_options: A tiny library for validating and documenting high-level options. 💽 which is kind of related here.

1 Like