Regarding deployments strategies (for zero downtime)

Hello,
Until now I was using docker container to build and deploy my phoenix apps.
Something as following:

  • I have a dev.Dockerfile and a docker-compose.dev.yml dedicated for development
  • I have a prod.Dockerfile and a docker-compose.yml dedicated to production
  • The reason I have 2 Dockerfiles (despite I heard that it’s not very good) is because the dev one has some development dedicated tools and the prod one is a multistage Dockerfile with a build stage (where I use Releases) and a very lightweight actual image (with only the Releases copied over).
    The compose files are barely different, mainly ports and envs.
  • To deploy, I SSH into the prod server, pull the repo and run docker-compose up -d .

So, nothing too fancy.
And in the meantime I even started to bring into play some CI/CD with Gitlab CI, and it seems even easier and cleaner to deploy, but the workflow is the same.

Now I wanted to tackle the downtime issue.

At first I thought about solving this “manually”. I mean like the green/blue strategy by having a load balancer and managing the replacement of the old image manually (here is a good blog post I find about the subject https://coderbook.com/@marcus/how-to-do-zero-downtime-deployments-of-docker-containers/)

Another solution seems to be using docker swarm abilities (with docker services) to update containers. I just heard of it and it seems to be a pretty interesting solution. This could also allow me to gradually get rid of docker-compose.

The other solution I want to discuss here, is using Elixir/Erlang own abilities to perform code reloading.
But first I want to say that I’m completely lost about the current tools and the “best way” to do it. I heard of distillery, edeliver, and also elixir releases. I don’t get what’s doing what.

Elixir Releases seems to be a recent tool and it seems that it’s not so spread.
In fact I’m already using releases but only because I had some hard try to figure out in my docker containers approach how to handle my env variable. Following some recipes I found, I’m mostly using it because I put all my env variables within the config/releases.exs file and it just works.

Edeliver seems to be a tool that automates what we could do manually otherwise like SSH’ing into the prod server and copying the files.

Distillery, I don’t get its usage.

So, I wanted to know more about this topic.

Where Elixir/Erlang intrinsic hot code reloading abilities come into play?

Also, is it possible to have docker containers and those Elixir/Erlang code reloading at the same time?

Thank you for all your insights about this topic.
I’ll be happy to hear more about your workflows.

Have a good day…

3 Likes

Releases are a self contained package of files for running a beam project in production. Both mix release as well as distillery are tools to build a release based on a mix project (mix release was mostly build by the learnings of the older distillery). Today most people seem to use mix release over distillery for being included in elixir and the consolidation of configuration options, but distillery has better support for hot code updates as it generates appup/relup files, while for mix release you’ll need to create them manually.

Edeliver as you’ve gathered is a more high level tool wanting to handle the whole deployment process, where building a release is just one piece.

As for docker: You can do hot code updates with docker (gigalixir does support that afaik), but it goes kinda against the grain of docker and the idea of immutable images/throwaway instance. Basically you’ll need to modify your docker instance while it’s running.

2 Likes

I guess that you’re right…

Particularly I found this particular chapter in the docs https://hexdocs.pm/mix/Mix.Tasks.Release.html#module-hot-code-upgrades where doing so is not encouraged.

So I guess I’ll stick with mix release and docker for generating new images and then use the methods used for green/blue deployment… I assume it’s a common way to do it after all so I’ll be able to find some resources to read…

The main thing to note here is that, once your release is running inside a container, doing blue green deployments with Elixir is basically the same as doing blue green deployments with any other language. Therefore, don’t limit the material you read to only material that talks about Elixir specifically. Pick one of the various ways to deploy applications with docker and use whatever built in blue green deployment mechanism it has.

2 Likes

Wouldn’t it work without modifying the instance if the release was mounted instead of being kept in the image?

1 Like

I guess so, but even just mounting the app into the running container is as much against the idea of immutable images as changing the files inside it would be.

1 Like

Well, technically the image remains immutable :smiley: but I agree that it goes against the grain of how docker images are typically used.

1 Like

Yeah, it’s for sure no longer an immutable image of your application, but at best an immutable image of your apps running os.

1 Like