Bundling scripts (mix ecto.setup etc) with new elixir releases

Questions / Help
,
1

I’ve done this a lot when using distillery - for example, bundling a script which will execute an ecto.migration task.

Here’s an example of with this kind of configuration.

The reason this is important is for kubernetes. Kubernetes has various lifecycle hooks such as healthcheck, startup, etc.

It’s very nice to provide the cluster administrators with the means to run extra tasks - and it’s even nicer when you package this with the application so that you have targets like this :

docker run --rm -ti \
                -e POSTGRES_HOSTNAME=postgres.notifications \
                -e POSTGRES_PASSWORD=postgres \
                -e POSTGRES_DATABASE=notifications_dev \
                -e DATADOG_AGENT_HOSTNAME=localhost \
                -e DATADOG_AGENT_APM_PORT=8126 \
                -e EXQ_HOST=redis.notifications \
                -e EXQ_PASSWORD= \
                -e EXQ_PORT=6379 \
                -e EXQ_NAMESPACE=exq \
                --name notificatons \
                --network notifications \
                --hostname notifications  \
                -p 4000:4000 \
                foo/notifications:latest help
Usage: notifications COMMAND [ARGS]

The known commands are:

    start                  Starts the system
    start_iex              Starts the system with IEx attached
    daemon                 Starts the system as a daemon
    daemon_iex             Starts the system as a daemon with IEx attached
    eval "EXPR"            Executes the given expression on a new, non-booted system
    rpc "EXPR"             Executes the given expression remotely on the running system
    remote                 Connects to the running system via a remote shell
    restart                Restarts the running system via a remote command
    stop                   Stops the running system via a remote command
    pid                    Prints the OS PID of the running system via a remote command
    version                Prints the release name and version to be booted
--> run_migration          Run ecto migration
--> update_email_templates Update email templates (for example)

ERROR: Unknown command help
2
4

Not getting a lot of responses - might need to go back to using distillery

5

So I actually read the docs properly - and there is some description given to adding custom tasks -https://hexdocs.pm/mix/Mix.Tasks.Release.html#module-one-off-commands-eval-and-rpc

What I’m wondering though, is what’s the idiomatic way to package them up nicely with the bin/ init script? Is there a standard extension point?

6

You can define them in lib. Here is an example from Phoenix release guides: https://hexdocs.pm/phoenix/releases.html#ecto-migrations-and-custom-commands

2 Likes
7

Apologies - I’ve communicated appallingly badly, so what I’d like to do (end user ergonomics/tight integration) is build in such a way that rather than the end user invoking:

$ _build/prod/rel/my_app/bin/my_app eval "MyApp.Release.migrate"

The end user invokes:

$ _build/prod/rel/my_app/bin/my_app migrate

I should have asked is there an extension point or idiomatic way to extend the startup script.

8

Ah, ok! No, there isn’t a way today to extend the built-in script. We considered this but we are unsure if this is really the direction we want to go. For example, wouldn’t it be better to have separate script in bin/? Or maybe, if the goal is to provide an API for users with no experience of Elixir, have a separate script altogether?

9

That‘s what I‘ve gone with:

2 Likes
10

A script in bin for running mix ecto.setup ? Apart from hard coding, how would it know where the application code, etc lived?

11

@bryanhuntesl to be clear, I mean it could be called bin/setup-db and inside it does ./my_app eval "..." and that’s all. :slight_smile:

2 Likes
12

That’s a pragmatic solution - I’ll give it a go - I was getting too dogmatic there - still, I’d consider distillery’s approach just from the perspective of UX.

13

This stuff is pretty ugly but I works - so I’m going to include it here in case anyone else needs to set this up for kubernetes pod initialization…

mix.exs :

defmodule Examples.Mixfile do
  use Mix.Project

  def project do
    [
      app: :examples,
      version: "0.0.1",
      elixir: "1.10.1",
      elixirc_paths: elixirc_paths(Mix.env()),
      compilers: Mix.compilers(),
      start_permanent: Mix.env() == :prod,
      aliases: aliases(),
      deps: deps(),
      releases: [
        examples: [
          steps: [:assemble, &copy_extra_files/1]
        ]
      ],
      test_coverage: [tool: ExCoveralls]
    ]
  end

  defp copy_extra_files(release) do
    dst = release.path <> "/bin/"

    ["create-db.sh", "migrate-db.sh", "setup-db.sh"]
    |> Enum.each(fn f -> File.cp!("rel/" <> f, dst <> f) end)

    release
  end

 .... ....
end

rel/create-db.sh :

#!/bin/sh

/opt/app/bin/examples eval "Application.load(:examples);  Confex.resolve_env!(:examples); dbconf = Confex.fetch_env!(:examples, Examples.Repo); Application.put_env(:examples, Examples.Repo, dbconf); [ :crypto, :logger, :ssl, :telemetry, :postgrex, :ecto ] |> Enum.each(&(Application.ensure_all_started(&1))); IO.puts(inspect(Examples.Repo.__adapter__.storage_up( Examples.Repo.config)));"

rel/migrate-db.sh :

#!/bin/sh

/opt/app/bin/examples eval "Application.load(:examples);  Confex.resolve_env!(:examples); dbconf = Confex.fetch_env!(:examples, Examples.Repo); Application.put_env(:examples, Examples.Repo, dbconf); [ :crypto, :logger, :ssl, :telemetry, :postgrex, :ecto , :ecto_sql ] |> Enum.each(&(Application.ensure_all_started(&1))); Examples.Repo.start_link ; Ecto.Migration.Supervisor.start_link ; Ecto.Migrator.run(Examples.Repo, Application.app_dir(:examples, \"priv/repo/migrations\"), :up, [all: true])"

rel/setup-db.sh

#!/bin/sh

/opt/app/bin/create-db.sh
/opt/app/bin/migrate-db.sh
2 Likes
14

Elixir v1.10 supports overlays, so you can also just drop those scripts in rel/overlays/bin/ without changing your mix.exs. :slight_smile:

4 Likes