Lets encrypt renewal & runtime config

Hi,

I have set up a VPS, installed certbot and managed to run Phoenix 1.5.7 using HTTPS.
The paths to the key and cert files from lets encrypt are read by phoenix from ENV variables in the prod.secret.exs file.
The files will be renewed automatically by certbot every 3 months using a cron job.

I can’t find answers to the following questions:

  • How to deal with the renewal of the keys on the Phoenix side? Do I have to restart the Phoenix server or will Phoenix read the updated files in due time?

  • Would the situation be the same with the new elixir runtime config files?

I would appreciate any insights on these issues which are new to me.
Thanks in advance for your help.

1 Like

While I don’t know the exact answers to your questions, maybe you can find some insights from this library.
And if you would accept bringing a dependency to your app, you could use the library to manage your certificates and eventually ditch certbot.

3 Likes

I keep my certs in /etc/letsencrypt/live/..., set in a docker env file.

I add this to my endpoint:

  plug Plug.Static,
    at: "/",
    from: "certbot/"

and have a matching rel/overlays/certbot folder (it’s empty, just exists to create the folder).

I have this in my phoenix service in my docker compose file:

    volumes:
      - ${LETS_ENCRYPT_ROOT}:/etc/letsencrypt:ro
      - ./certbot:/app/certbot

I use certbot’s webroot option and basically tell it to go use /my/compose/root/certbot, which gets mounted at /app/certbot, which cowboy will serve at /, and it all just works.

First run you have to shuffle some mess around but it’s easier to use certbot’s own webserver to generate the first set of certs, then change it to use webroot.

The SSL guide recommends you use the fullchain.pem file for atomic updates, you don’t have to restart phx.

1 Like

I use traefik running in front of my app. It cares about the certificate renewal and also routing.

2 Likes

I am also using it for now, but be very careful with trusting in the headers because they don’t cleanup the headers as they should. For example, the x-forward-for is set by them, but it can also be set by the client sending the request, thus you end-up with two ips in your backend:

curl -I --header "X-Forwarded-For: 82.22.85.84" https://your-site-behind-traefik.tld

Your backend logs:

app_1        | 18:51:33.229 | info | HEAD / | request_id=FnZrTLGwHk701AMAAAax proxy_ip=82.22.85.84, xx.23x.8x.8x remote_ip=::ffff:172.18.0.2 user_agent=curl/7.68.0 module=Phoenix.Logger function=phoenix_endpoint_start/4 line=178

So which IP you gonna trust now proxy_ip=82.22.85.84, xx.23x.8x.8x?

It seems that the last is the real IP from the client, but Treafik should always drop any header it sets, as any good behaved proxy does :slight_smile:

1 Like

That is true, overwritten headers could be an issue. But maybe it is not an issue for @damien I find it nice not to care about certificate renewal.

Regarding the overwritten headers… what if you configure traefik / your LB to use some other, “unforgeable” name for the header, e.g. X-Forwarded-34535435987-For? :slight_smile:

1 Like

Could be a workaround if Traefik allows for that, but then you also need to account for that in software that assumes the standard header for the proxy to be the x-fowarded-for.

I was loving Traefik until I discovered this issue and that they kept “refusing” to fix it, despite open bugs.

So, I will have to ditch Traefik in a near future, because if they screw-up in the basics of security and won’t fix I don’t even want to imagine what else they got fundamentally wrong.

I strongly recommend you to use the library mentioned by @crova:

This library is from @sasajuric the author of the book Elixir in Action and also a very reputable member of our community :slight_smile:

If those files are updated, I think you don’t need to restart anything and the new certs will be eventually picked up. FWIW site_encrypt forces this manually by invoking :ssl.clear_pem_cache.

5 Likes