Hi people,
Basically the topic says it - what is your rollback strategy for full OTP releases (not “upgrades”)?
Is it possible to swap release built with Distillery?
1 Like
Are you talking about rollbacks for an application written and deployed using Distillery, or are you asking about rolling back the OTP system itself (i.e. roll back to OTP 20 from OTP 21 for example)?
2 Likes
It would depend on how you deploy…
Most of my deployments lately have been .deb packages generated from a full release. In that scenario, I’d stop the app, roll back any database migrations if needed, reinstall the older version from the previous .deb package, and start the application again.
If you’ve just been extracting .tar files, I think each full release automatically overwrites the startup scripts etc, so you may have to re-extract the .tar file from the previous release - or just fiddle around manually. If I recall correctly, your older release should be sitting in its own versioned directory as well.
@easco applications
@jwarlander I’ve gone the OS package road and now want to try to skip that layer and deal only with tar.gz files. You’re right - the releases are already in releases/ folder and I thought that it should be somehow possible to switch between them.
maybe @bitwalker can give a tip 
We use distillery back at my place. We have a releases server (that saves all releases of a given app) and a deploy server. Since we are working with distillery every release is a tar.gz file, so the releases server is basically a backup of the latest 10 releases or so.
If we need a rollback, we simply ask the deploy server (which deploys a given tar.gz to the production machines) to deploy the previous release of a given app. Nothing could be simpler.
1 Like
How do you handle rolling back the migrations?
We are using the rolling replace method for our apps. So imagining we have 10 servers, we take down 3 and spin up 3 with the newest version. We find this is the cleanest way, as everything starts fresh.
As for DB migrations (is this what you are actually asking?) there is no simple answer. Once a bad state is persisted in a DB, there is little you can do. What we do, is we delete the state that was persisted with the newest version of the app, and then (because all requests hit our balancer before being processed by our apps) we query the balancer for the latest X hours or days and we re-feed those queries to the stable version of the app.
It’s not 100% accurate, but we find it works well enough for us. After all, if you mess up a deploy, there is always a cost.
