What is your strategy for running one-off and automated tasks using releases (no mix)?

When deploying your Phoenix app via Elixir releases, how do you manage your database migrations? (and any other automated tasks). How do you run one-off tasks?

releases come with:

eval "EXPR"  Executes the given expression on a new, non-booted system
rpc "EXPR"   Executes the given expression remotely on the running system

I’m mostly fine with these if I’m migrating the database but ideally I’d love more structure than calling the code directly. I especially want more structure for one-off tasks. Shell scripts are an option but I’m not sure what the community recommends.

I came across this article, http://blog.firstiwaslike.com/elixir-deployments-with-distillery-running-ecto-migrations/ but it was written some time ago so am not sure if it still holds.

Any thoughts would be greatly appreciated. Thank you

Migration supervisor child.

children =
      [
        libcluster_child(),
        Sensetra.Endpoint,
        {Absinthe.Subscription, Sensetra.Endpoint},
        Sensetra.Repo,
        Sensetra.Repo.Migrator,
        # ... other children
        Sensetra.Ready
      ]

Endpoint is first so that there is a working /alive endpoint for liveliness checks. Then we boot the repo, run migrations, other children, and ultimately Sensetra.Ready runs to enable readiness checks.

Here is the migrator code:

defmodule Sensetra.Repo.Migrator do
  use GenServer
  require Logger

  def start_link(_) do
    GenServer.start_link(__MODULE__, [], [])
  end

  def init(_) do
    migrate!()
    {:ok, nil}
  end

  def migrate! do
    path = Application.app_dir(:sensetra, "priv/repo/migrations")

    Ecto.Migrator.run(Sensetra.Repo, path, :up, all: true)
  end
end

I wonder if ecto_sql should include a child for this. Then you could have:

{Ecto.Migrator, MyApp.Repo}

as a child.

EDIT: Created https://groups.google.com/forum/#!topic/elixir-ecto/Iraj2MbDwLg

I should add that this style of running migrations assumes the following conventions about migrations:

  1. Database migrations are always backwards compatible. That is to say, after the migrations have run, the previous version of the code should still work. This is I think critical both because in rolling deploy scenarios you have two versions of your code running simultaneously, and also because if your new code has issues you make it much easier to roll back. You don’t need to worry about rolling back the database, just the code.
  2. New code is allowed to depend on the migrations. Because we can guarantee that the migrations happen at the right point in the supervision tree, code that runs later is allowed to use the new database changes. If the migrations won’t succeed, your new app won’t boot and won’t be routed any traffic.

When you need / want to do backwards incompatible database migrations you do so in two phases. You first roll out a version of the database schema that is backwards compatible with the old code, and contains new code that no longer depends on the stuff you want to get rid of. Then you do a second deployment after that has succeeded where you eliminate the stuff you are getting rid of, because this is now a backwards compatible change.

5 Likes

Thank you for the detailed explanation. Sorry for the silly question… if i have one app running and one (small–ish) db, do I need a GenServer to run the db migrations?

The main reason I think it’s still worthwhile: You can’t forget to do them. Plus, there’s no moment of time where the new code is running and the migrations haven’t run.

When you say one app running do you mean just one instance of your app?

1 Like

Yes, I mean one instance.

Thank you so much for the help!