arcanemachine

arcanemachine

Preamble: I am aware of other methods of dynamic runtime configuration (e.g. Config.Provider), and am currently a happy user of Dotenvy. The title of this thread is not “How to dynamically configure runtime config?”, but is specifically pertains to the usage of Code.eval_file/1 as a strategy to do so.

I am experimenting with different strategies to dynamically import runtime config, similar to how Config.import_config/1 works for compile-time config.

I have come up with a solution that appears to work. I am bringing the topic here for discussion, to see if there are any valid technical reasons why I should not use this strategy.

The strategy

Here is an example config/runtime.exs that uses Code.eval_file/1 to import a local runtime config file, if one exists:

config/runtime.exs

import Config

config :my_project, :hello, :world

# Maybe import local runtime config file(s)
for local_runtime_config_file_name <-
    ["runtime.local.exs", "runtime.#{config_env()}.local.exs"] do
  local_runtime_config_file_path = Path.join(__DIR__, local_runtime_config_file_name)

  if File.exists?(local_runtime_config_file_path) do
    IO.puts("Importing local runtime config from `#{local_runtime_config_file_path}`...")

    Code.eval_file(local_runtime_config_file_path)
  end
end

This allows one or more (presumably gitignored) local runtime config files to be added, without having to modify the main runtime config file:

config/runtime.local.exs

import Config

config :my_project, :hello, :another_world

The above example shows how a local runtime config file could be loaded for 1) all configuration environments, and/or 2) a specific configuration environment (e.g. :dev, :test, etc.), and supports cascading (i.e. the global runtime.local.exs would be overridden by e.g. runtime.dev.local.exs). If any of the local files do not exist, then that is fine also, and no custom config would be loaded.


I have this running, and it appears to work. But I haven’t seen this strategy documented anywhere, so I’m a little wary of using it.

Does this appear to be a viable strategy, or is there some hidden footgun here that I am not aware of?

Showing Posts 1 to 8

mudasobwa

mudasobwa

Creator of Cure

I might be missing something, but what would be wrong with plain old good Config.import_config/2?

for config <- ["runtime.local.exs", "runtime.#{config_env()}.local.exs"] |> Path.expand(__DIR__) |> Path.wildcard() do
  if File.exists?(config), do: import_config(config)
end
tfwright

tfwright

I infer from your preable that you are specifically looking for technical answers, but since you mention this purpose I am curious why in such a situation you wouldn’t just use a single default “example” file and modify/copy it to the “real location” as needed, which I think is a pretty standard way to handle this kind of thing?

I suspect any real answer to your question would benefit from similar details, because answers to a question like “can I” generally come with a lot of “depends”…

LostKobrakai

LostKobrakai

Instead of answering the question I’ll answer why this isn’t a feature in the first place:

Runtime config needs to be copied into releases, so the config can be executed when the application boots. That copying means mix needs to know which files to copy in the first place statically – it cannot run the config file to figure it out, because conditions might change. Therefore to keep things easy elixir limits runtime config to a single file by default, which has to exist with a known name at a known location. All this happens under the premise that a release once assembled can be copied as is and is startable.

Now you can manually make sure additional files are also copied into the release and then you could read them. It would retain all the properties elixir/mix tries to uphold from it’s pov knowling less details about your project. I’ll leave the judgement if that’s a good idea to yourself though.

arcanemachine

arcanemachine OP

I thought that didn’t work for runtime config?

I need to look into this again…

EDIT: Got this error: ** (RuntimeError) import_config/1 is not enabled for this configuration file.

arcanemachine

arcanemachine OP

We basically do what you’re describing, but it can lead to drift between the committed runtime.exs file and what the DevOps people’s file looks like. There have been times where a dev committed code that was in their local compile-time config, but wasn’t actually committed to the code repo, which can lead to confusion when other people pull the code and have no way of seeing what the config value should be, leading to some confusing crashes. (I realize that this is not a problem with the runtime config itself, but it sort of addresses an issue with the example runtime.exs file, which is that “drift” can occur.)

What it really comes down to is this: When I use Dotenvy, I get a nice “base” runtime.exs, which I can extend for each environment with custom dotenv files. This also prevents the “drift” in runtime.exs that I mentioned before. I’m just trying to see if I can come up with a solution that feels similarly “right” without needing the external dependency. (I’m a huge Dotenvy fan because the same dotenv config files can also be used to customize a Docker image environment, which has been useful.)

mudasobwa

mudasobwa

Creator of Cure

Ah, indeed. The documentation even explicitly states that. Good to know.

LostKobrakai

LostKobrakai

Personally I skip file in the equation. All the things that need to be set from the outside at runtime need to come in as system env variables. runtime.exs reads those. But elixir/mix/releases are not at all involved at setting those variables. For that I tend to use direnv locally / in development and whatever tool my deployment target provides to set environment variables. Those may or may not be able to source from files committed to the repo itself - e.g. .envrc for direnv can be commited, but a systemd unit might live elsewhere – so first and foremost I want to provide documentation (and good errors) around the set of expected env variables, rather than trying to prescribe how downstream people have to provide values. The runtime.exs essentially becomes a mapping from system env input to elixir config values for the minimal set of what should be configurable at runtime.

adamu

adamu

It sounds like you have a local uncommited .exs file that is duplicating the role that runtime.exs in plays in a release. So in the uncommited .exs you are setting up config using elixir, and runtime.exs you are setting the same config via environment variables.

It might simplify things to remove the ability to have any uncommited local config, and have any customisations set via environment variables via runtime.exs locally, too. That way, if a dev forgets to commit the config, it will fail to compile on CI / for everyone else.

edit: whoops, posted before seeing the above post. I think we’re saying the same thing.

— All posts loaded —

Where Next? Top

Trending in Questions Top

Blokh
Hey guys, I’ve got a huge CSV ( around 10 GB ) that needs to be processed hourly Do you guys have any suggestions what is the best prac...
New
kszambelanczyk
Hello! Could someone please give me a help/sample code, how to delete a file from s3 using waffle/waffle_ecto from Phoenix app. I creat...
New
RemyXRenard
I’m seeing that a list inside a Kino.DataTable will be interpreted as a charlist, even if the Kino.configure() is set to charlists: :as_l...
New
matt-savvy
Anyone here using Honeybadger? My Honeybadger account is being overwhelmed with noise from some bots. Seeing a lot of Bandit.HTTPError...
New
velrest
So my question is quite simple and i have found no conclusive answer on forum, google or AI. Should we use :erlang.float for Integer to ...
New
samoloth
Hi, I’ve just set up an application with ash_authentication. There is only magic link strategy for now, so there is no confirmation add o...
New
FlyingNoodle
If a change or preparation module uses Ash.Changeset.get_argument/2 or Ash.Query.get_argument/2 (or any of the other get_argument functio...
New

Other Trending Topics Top

mudasobwa
I am happy to introduce the very α version of the new programming language compiled to BEAM. Welcome Cure. It has literally three kille...
New
garrison
Hobbes is a low-level distributed database for the Elixir programming language. Hobbes provides a simple, safe, and scalable storage lay...
New
jimsynz
Beam Bots (or just BB for short) is a framework for building fault-tolerant robotics applications in Elixir using familiar OTP patterns. ...
New
mcass19
ExRatatui lets you cook up rich terminal UIs in Elixir, powered by Rust’s ratatui via Rustler NIFs. Build interactive terminal applicatio...
New
Damirados
Hello everyone. After busy few months I am happy to announce v0.1.0 of Emerge &amp; Solve. They are GUI (Emerge) and State management (S...
New
netoum
Corex is an accessible, unstyled UI component library for Phoenix that integrates Zag.js state machines using Vanilla JavaScript and Live...
New

We're in Beta

About us Mission Statement

Options

Thread Display Mode




Thread Preview

Skip Thread Previews