How to best upgrade a production Phoenix application

I build a Phoenix application with MIX_ENV=prod mix release on a Debian Linux system. It gets started with _build/prod/rel/animina/bin/animina start. An Nginx server is used as a reverse proxy.

The problem: Because of some AI stuff it takes very long to start the Phoenix application. I will try to move the AI stuff to a different application but right now I am stuck with it. So stopping the old and starting the new version of the applications becomes a problem because the service is not available for at least 45 seconds.

Is there a solution how to minimise the outage? Can I start the new one first and than kill the old one? Or is hot deployment possible?

You can introduce some means of orchestration and readiness/liveness probes to allow the orchestrator to check if the application is ready to serve.

And only initial readiness is signalled, switch the reverse proxy to serve the new app, then shut down the old one.

As long as there are no incompatible database changes this technique usually works well, assuming any other ressource can also be shared between the services.

1 Like

Yeah, this works no different with phoenix than it does with other systems. Blue green deployments can be accomplished with lots of tools you might be using to start a release. You need some way to separate routing from the applications lifecycle though to be able to start a new instance and only later switch the routing to said new instance.

4 Likes

If you’re deploying to a single host, one concrete solution is installing something like Dokku on the host. It manages an nginx proxy (or others you can choose) in front of your app and allows for easy git push based zero-downtime deployment, by starting a second copy of your app, waiting until it’s ready, switching traffic to it, and then draining connections from your old container and cleaning it up.

Getting Started with Dokku - Dokku Documentation

3 Likes