Some environment info
We are hosting our Phoenix application on a Kubernetes cluster using a Dockerfile based on the one in distillery’s documentation.
We are using peerage to discover other nodes of this application, which solves node discovery when dynamically scaling the cluster up or down. This means that the nodes will know about each other so that Phoenix Channels can send events to all nodes.
Other perhaps useful info?
There’s a CICD pipeline for each dev, beta and prod, where in dev we’d want migrations to run automatically while in beta and prod we might want other settings/steps for this.
Hooks don’t work
The deal is though that the previous solution to running ecto migrations was to have a pre-start hook in the rel/config.exs
such that it would run these migrations before starting the application. This will however run them each time a node starts, or if they are restarted. Which is not what I want.
We do not want running migrations to be part of any boot-up sequence to minimize boot-up times and to also make sure that we can control it manually, should we need to.
Goal
Find a flexible and easy way to run ecto migrations without adding hooks in the release that would run on each pod start, slowing a pod’s start up time down.
Approach one - Escript
One approach I have considered would be to write an escript that could be compiled and installed locally to manage these tasks, which could then be used in our CD by running the script(s) before deploying the updated docker image.
Approach two - Admin feature
We could implement an internal API that’s restricted to devs where one can start migrations, rollback or whatever. However, it will require that I deploy before I actually can migrate the newest changes, which would mean that until it is deployed and somebody then starts the migrations, there has to be some kind of precautions in all the endpoints checking if migrations has been done, as there might be missing fields in the database before this. Which is why I would prefer the first approach.
Combined
One could also combine these approaches for different cases
Current solutions?
What are the current solutions people use? Are they adaptable to a cluster environment where you don’t always know the host machine (or at least shouldn’t have to know it)?