Exrm migration problem

Hello,

I’m using exrm with Docker and was trying to use the “new” command feature.
I’ve read this blog post and found these gists 1, 2

And ended up with something that I thought should/could work.
But unfortunately it does not.
But the error that I’m getting is very weird, and I’m a bit lost.
Can someone help me?

The error I’m getting is:

{:compiler, {'no such file or directory', 'compiler.app'}}

I can share a gist with more information if someone has an idea.
The error occurs once when I run the following command:

rel/my_app/bin/my_app command release_tasks migrate

and the code where the error occurs is the following:

{_ , message } = Application.ensure_all_started(app)

Thanks a lot.

Mila

2 Likes

I’m using exrm in Digital Ocean and migrations are working. Is your release starting/stopping properly? You should have compiler directory inside /rel/your_app/lib/

3 Likes

If you can share your mix.exs, the :release_tasks module and the commands you use to build your release, it might be easier to quickly spot if something seems out of place :slight_smile:

2 Likes

thanks for your answers.

I replaced all occurences of the app name with MyApp so I hope I did not miss anything.

the command to build the release is the following:
MIX_ENV=prod mix compile and MIX_ENV=prod mix release

trying to execute the following command: rel/my_app/bin/my_app command sync_release seed

here is my mix file:

defmodule MyApp.Mixfile do
  use Mix.Project

  def project do
    [app: :my_app,
     version: "0.0.24",
     elixir: "~> 1.2",
     elixirc_paths: elixirc_paths(Mix.env),
     compilers: [:phoenix] ++ Mix.compilers,
     build_embedded: Mix.env == :prod,
     start_permanent: Mix.env == :prod,
     deps: deps]
  end

  # Configuration for the OTP application
  #
  # Type `mix help compile.app` for more information
  def application do
    [mod: {MyApp, []},
     applications: [:phoenix, :phoenix_html, :cowboy, :logger,
                    :phoenix_ecto, :postgrex, :remodel, :mailgun,
                    :geo, :ex_csv, :scrivener, :cors_plug,
                    :timex, :uuid, :ssl, :erlcloud, :csv,
                    :honeybadger]]
  end

  # Specifies which paths to compile per environment
  defp elixirc_paths(:test), do: ["lib", "web", "test/support"]
  defp elixirc_paths(_),     do: ["lib", "web"]

  # Specifies your project dependencies
  #
  # Type `mix help deps` for examples and options
  defp deps do
    [{:phoenix, "~> 1.1"},
     {:phoenix_ecto, ">= 1.0.0"},
     {:postgrex, ">= 0.0.0"},
     {:phoenix_html, "~> 2.1"},
     {:phoenix_live_reload, "~> 1.0", only: :dev},
     {:cowboy, "~> 1.0"},
     {:mailgun, "~> 0.1.2"},
     {:remodel, "~> 0.0.1"},
     {:ex_csv, "~> 0.1.3"},
     {:csv, "~> 1.1.5"},
     {:geo, "~> 1.0"},
     {:exq, "~> 0.4"},
     {:scrivener, "~> 1.1"},
     {:cors_plug, "~> 0.1.4"},
     {:erlcloud, "~> 0.12.0"},
     {:honeybadger, "~> 0.3"},
     {:exrm_docker, "~> 0.1"},
     {:exrm, "~> 1.0.4"}]
  end
end

and my release_tasks module

defmodule :sync_release do
  def seed do
    repo = MyApp.Repo
    start_repo(repo)
    info "Seeding data for #{inspect repo}.."

    #seed
    
    :init.stop()
  end

  def migrate do
    repo = MyApp.Repo
    start_repo(repo)

    path = Application.app_dir(:my_app, "priv/repo/migrations")
    Ecto.Migrator.run(MyApp.Repo, path, :up, all: true)

    :init.stop()
  end

  defp start_applications(apps) do
    Enum.each(apps, fn app ->
      {_ , message } = Application.ensure_all_started(app)
      info "Start application #{inspect app} got message: #{inspect message}"
    end)
  end

  defp start_repo(repo) do
    load_app()
    info "Start Repo"
    message = repo.start_link()
    info(inspect message)
  end

  defp load_app do
    start_applications([:logger, :postgrex, :ecto])
    info "Loading App"
    :ok = Application.load(:my_app)
  end

  defp info(message) do
    IO.puts(message)
  end

  defp fatal(message) do
    IO.puts :stderr, message
    System.halt(1)
  end
end

@defoemark are you sure I need the compiler inside the rel/my_app/lib/ folder ?

2 Likes

seems like it should be compiler here, since you get this error. Try to put {:ok, _} = Application.ensure_all_started(:my_app) at the top of migrate.

2 Likes

@milafrerichs, I tried creating a “clean” Phoenix app and just pulling in ExRM + setting up the :sync_release module. Building a release from this, and running the seed task, seems to work fine:

$ mix help
...
mix phoenix.new       # Creates a new Phoenix v1.1.4 application
...

$ elixir -v
Erlang/OTP 18 [erts-7.3] [source-d2a6d81] [64-bit] [smp:4:4] [async-threads:10] [hipe] [kernel-poll:false]

Elixir 1.2.4

$ mix phoenix.new my_app
* creating my_app/config/config.exs
* creating my_app/config/dev.exs
* creating my_app/config/prod.exs
...
Fetch and install dependencies? [Yn] y
* running mix deps.get
* running npm install && node node_modules/brunch/bin/brunch build
...

$ cd my_app
$ vim mix.exs
$ vim lib/sync_release.ex

$ MIX_ENV=prod mix compile
==> connection
Compiled lib/connection.ex
Generated connection app
...
Consolidated Enumerable
Consolidated IEx.Info
Consolidated Inspect

$ MIX_ENV=prod mix release
Building release with MIX_ENV=prod.
==> The release for my_app-0.0.1 is ready!
==> You can boot a console running your release with `$ rel/my_app/bin/my_app console`

$ rel/my_app/bin/my_app command sync_release seed
mkdir: created directory ‘/home/johwar/git/my_app/rel/my_app/running-config’
Start application :logger got message: [:compiler, :elixir, :logger]
Start application :postgrex got message: [:connection, :db_connection, :decimal, :postgrex]
Start application :ecto got message: [:poolboy, :ecto]
Loading App
Start Repo
{:ok, #PID<0.75.0>}
Seeding data for MyApp.Repo..

Have a look at the code here:

2 Likes

I’ll note that compiler is indeed in the release libs:

$ ls rel/my_app/lib/
compiler-6.0.3    cowboy-1.0.4  crypto-3.6.3         decimal-1.1.2  eex-1.2.4     gettext-0.11.0  kernel-4.2    my_app-0.0.1   phoenix_ecto-2.0.2  plug-1.1.4    poolboy-1.5.1    ranch-1.2.1  stdlib-2.8
connection-1.0.2  cowlib-1.0.2  db_connection-0.2.5  ecto-1.1.7     elixir-1.2.4  iex-1.2.4       logger-1.2.4  phoenix-1.1.4  phoenix_html-2.5.1  poison-2.1.0  postgrex-0.11.1  sasl-2.7
2 Likes

thanks. that’s very helpful.
compiler is not in my release folder. Will try to generate a clean release and check back.

2 Likes

even after a clean compile and release the compiler package is not there.
where does this come from?

2 Likes

ok, after a bit of digging into it.
the compiler package is there if you do not include {include_erts, false}. in relx.config
but I don’t want to include the erts since I’m building on OSX and want to deploy to ubuntu.

2 Likes

You seem to be on a particular debugging track. Hopefully that solves your problem :slight_smile:

I’ll mention that command is pretty new, and has been through some fixes. Strongly recommend running the latest, 1.0.5 as of two days ago…

~m

1 Like

My ideal process if I were in that situation, would probably be to build in an Ubuntu VM (or container), and include the erts as well… That way, the release itself does have all of the dependencies required, and nothing more is needed from the OS setup in the target environment.

I seem to recall having seen some automated way of doing this with Docker containers… just can’t remember where :slight_smile:

yeah, this is my ideal solution as well. will probalby use CircleCI to build the release once the tests are all green and then push to the registry and have the container pull the latest image.

but not there yet, need to make sure the migrations etc are working with a command.

thanks @martin-langhoff it seems to be working with the new release of exrm
but will dig a little deeper and report back.

it worked indeed with the new exrm version.
but the error message is very odd.

There’s been all sorts of odd errors with versions prior to 1.0.5.

In addition, you need to start the apps your maint functions will need, without starting your whole app. After a few kludges, we settled on the pattern described here https://github.com/bitwalker/exrm/issues/67#issuecomment-221718993

If you search the exrm issues (regardless of status), looking for command, you’ll find a bunch of good recipes and debugging tricks :slight_smile: – need to be pulled into the docs…

ok thanks. do you have a slack or IRC channel where you hang out? or the other contributors to exrm?

having trouble with postgrex connection with the release. or better with ssl.

in the mornings I hang out on irc. freenode, #elixir-lang . bitwalker is there sometimes.

Are you using conform?

m

yes, I’m using conform. SSL true is set in the configuration and added to the conf file via conform.

Getting this error:

[gapi-f233ea6f-1]2016-05-26T13:58:13.186307105Z {"init terminating in do_boot",{{undef,[{crypto,supports,[],[]},{tls_record,supported_protocol_versions,1,[{file,"tls_record.erl"},{line,412}]},{tls_record,supported_protocol_versions,0,[{file,"tls_record.erl"},{line,323}]},{ssl,handle_options,2,[{file,"ssl.erl"},{line,676}]},{ssl,connect,3,[{file,"ssl.erl"},{line,105}]},{'Elixir.Postgrex.Connection',handle_info,2,[{file,"lib/postgrex/connection.ex"},{line,277}]},{gen_server,try_dispatch,4,[{file,"gen_server.erl"},{line,615}]},{gen_server,handle_msg,5,[{file,"gen_server.erl"},{line,681}]}]},{'Elixir.GenServer',call,[<0.95.0>,{query,<<"BEGIN">>,[]},5000]}}}

That is a common error. When running under Mix, all your deps get started for you. Running under a release, you must add them explicitly to applications.

In this case, the {{undef,[{crypto,… is the hint you need. Add crypto to your applications list…

This is listed as a common error in the README and docs for exrm. There’s a few other common errors and debugging tricks you might want to check upon…

~m