Managing env. variables in a simple way and deploying them on a VPS

Questions / Help
1

Let’s say I have a bunch of enviromental variables in a project. I want to accure that I won’t forget to set on a VPS any of those when I deploy my project. Instead of seaching for them accross a project one by one and then setting them on a VPS manually, is there a way to simplify this process? By generating a file with them that I’ll then use on a VPS to set them up in one fell swoop. Or something similar. How is this done usually?

I’ll prefer a simple solution, with no third-party libraries.

2

Do you have any existing tooling or automation used for deploying the application or provisioning your VPS’s? Could you tell us a little more about your infrastructure?

2 Likes
3

Pending the answer to Louis’ request for more context:

If you’re hosting on a relatively recent Linux distribution that uses systemd, and you’re running directly on the host rather than through containers or another mechanism that might already have facilities for managing this, you could use an EnvironmentFile directive in your systemd unit. Then whatever technique you use to ship the code to the deployment targets should write/rewrite that file. You can simulate using the same content locally during development with something like direnv.

Generally my position as well! But in this specific instance, most of the prior art (other than a dotenv Hex package that has a big warning on its README) is rather good, and shouldn’t be dismissed out of hand.

1 Like
4

I’m not asking how to set env. variables. I’m asking “how to gather all of them in a single place so that I can see all of them before deploy”

5

The way we do it is to have the code in our application only depend on settings in the configuration files.
These settings are then filled in by defaults on the development environment, but are fetched from environment variables in production.

Because all environment variable names are now mentioned in the production configuration, it is easier to see what needs to be configured.

In production, there are tools to make setting up the environment variables less error-prone. In our deployment process we specify a file that contains the environment variables in the application’s systemd service configuration (see that topic for more information and steps on how to do this). This does not require any extra tools, because systemd is already the thing that manages services by default on most Unix-machines.

1 Like
6

Depends on how and when you read them.

  • Are they all read in the config/*.exs? Look them up there.
  • Do you have runtime calls to System.get_env/1? Replace them with a macro, which at compile time creates file with the env variables read. This will only give you the full list though on a pristine build.
  • Do you use distilleries deprecated(?) REPLACE_OS_VARS? Skim the config files again.
  • Do you use distilleries config providers? Skim the appropriate provider.
  • Do you use more than one of the above? Combine solutions as necessary.

Last but not least, better document those right from the start in your next project. Also I have learnt the hard way, do not read config lazily at runtime. Do it in a central place at “boot” and have a central place to put it. On the BEAM the application environment is a good place to do so.

3 Likes
7

Why that assumption?

8

I do. Why replace, how will then I read them, if I replace them with a macro? Can you show me an example? Given that I may have them in different places accross a project.

9 
defmacro read_sys_var(var) do
  # Write value of var to a file, if it's representing a string
  quote do
    System.get_env(unquote(var))
  end
end

This is a rough concept draft. You need to find places where you call it to replace the direct call with the macro.

Also this will only work for compile time static string literals.

10

How will that help me versus having direct call to System.get_env(var)?

11

As I said already, if you write the comment part correctly, then you’ll have a file that lists the variables after a pristine build.

12

If you have a single file that lists all pieces of configuration information you need, then you can see what configuration settings your application requires in a single glance.
If calls to System.get_env/1 are sprinkled all through your application, then you will not be able to see this at a single glance. If you are lucky, a smart text-search will find all occurrences, but don’t count on it.

13

No. Why do you assume that I have them in the prod. config only?

14

Assume that you have what in the production configuration only?

I assume that you will configure each of your environments (development, testing, production; maybe others). Whether they are configured in all environments using environment variables, using hard-coded values or some other method does not matter (and might vary per setting).

15

You seem to disagree with each of the suggestions that people have offered - which is absolutely your prerogative, as its your project and your needs. However, you’re not sharing much specific/objective feedback about why those approaches are either unsuitable or incomplete, and you’ve been somewhat less than forthcoming about your own requirements/constraints and about what you’ve tried so far. Given that this whole exercise is a volunteer effort for each of the other participants in the thread, you can only expect so many attempts before people expect your answer will always be “no, I don’t like that” and move on. Good luck with your solution, I’m confident there is a suitable approach out there somewhere! :wave:

1 Like
16

I seem.