Structure code to run before any application starts

Questions / Help
,
1

Hello,

I have an umbrella application with many children applications. I would like to run some code (the same code) if any or all those child applications are started (i.e. when running tests, when running iex -S mix at the umbrella level, etc.) What would be the best way to structure such code?

Thanks!

2

Maybe you can explain exactly what you are trying to accomplish.

Nerves does something like this with the Shoehorn application, it ALWAYS starts, so even if other applications crash, you always have something running.

I’m not exactly sure how it accomplishes this, but you could dig into the code. https://hexdocs.pm/shoehorn/readme.html

But maybe there’s another way to accomplish what you need to do.

1 Like
3

Good idea, this could be an example of the XY problem. I am trying to store some DB rows in :persistent_term and want to guarantee that the rows will be stored in :persistent_term before any of the applications start.

And thanks for the link, will take a look!

4

If it wasn’t an umbrella application, you could just use a supervision tree, and have your first process insert the data, but assuming you don’t want to refactor everything :wink: Shoehorn might be exactly what you need.

At the same time, the idea of a global state being shared across all of your applications is kind of frightening.

What exactly is the use case?

5

Couldn’t you create a new application (that all of the other applications depend on), whose supervision tree contains a GenServer that inserts the rows in :persistent_term?

4 Likes
6

In addition to @axelson’s excellent suggestion, do also have a look at GenServer’s handle_continue.

1 Like
7

I have used the Module @on_load callback to ensure persistent terms are set up the first time the wrapper module is used. That allowed me to avoid any concerns about app startup processes.

2 Likes
8

That’s an interesting approach, but I am not able to get it to work with modules that represent a schema. I keep getting the error: __MODULE__.__schema__/1 is undefined (function not available). Do you know what might be causing it?

9

I suspect Ecto’s schema macro is setting an @on_load callback and yours overrides it. If you could find the right call, your callback could call Ecto’s

10

Thanks for the quick reply! Turns out I was not returning :ok like I was supposed to from the on_load function. Returning :ok solved the issue.

This approach looks promising. I am going to play around with it and report back.

11

Oh, good because I just grepped the ecto code and “on_load” does not appear, so my suspicion was all wrong :sweat_smile:

I think my first attempt forgot that :ok too, so I can relate. I hope it works out for you.

2 Likes
12

Unfortunately, the on_load callback won’t work in this case because the module itself is not loaded when the callback is run, so I can’t query using the module. I suppose I need to use something like after_load, but I don’t think it exists. Will take a look at some of the other suggestions in this thread.

Thanks for the pointer tho, I see it being useful in some other cases :slight_smile:

13

If you’re using an umbrella I found it useful to create an application that is responsible for bootstrapping the actual app. This allows you to setup things before starting your actual “core”.
As an example, on the umbrella mix.exs I have the following release:

defmodule YourApp.MixProject do
  use Mix.Project

  def project do
    [
      apps_path: "apps",
      version: "1.0.0",
      start_permanent: Mix.env() == :prod,
      deps: deps(),
      releases: [
        your_release_name: [
          applications: [
            bootstrap: :permanent, #this is so that when releasing it starts this app which in turn will run its logic
            server: :load #this is the actual core application, we set it to only load its modules but not start the supervision tree
          ],
          include_executables_for: [:unix]
        ]
      ]
    ]
  end
  #other things.... like deps etc
end

Then the bootstrap app is a simple app with a supervision tree, it has a bootstrap.ex which is a gen_server to be started from the supervision tree, eg:

defmodule Bootstrap do
  
  @moduledoc """
  This starts mnesia and etc and once ok starts the web interface
  """

  use GenServer, shutdown: 50_000
  require Logger
  
  @mnesia_tables_attrs %{
    categories: [:slug, :title, :description, :image, :id, :struct]
  }

  @mnesia_tables Enum.reduce(@mnesia_tables_attrs, [], fn({table, _}, acc) ->
    [table | acc]
  end) |> :lists.reverse

  def start_link(_) do
    GenServer.start_link(__MODULE__, nil, name: __MODULE__)
  end

  def init(_) do
    System.cmd("epmd", ["-daemon"])

    #random_uuid = (:crypto.strong_rand_bytes(4) |> Base.encode16())
    #name = :"#{random_uuid}@#{:net_adm.localhost()}"

    #:net_kernel.start([name])
    
    case :mnesia.start() do
      :ok ->
        
        Enum.each(@mnesia_tables_attrs, fn({table, attributes}) ->
          :mnesia.create_table(table, [attributes: attributes])
        end)
        
        case :mnesia.wait_for_tables(@mnesia_tables, 5_000) do
          :ok ->
            Logger.warn("Mnesia tables loaded")
            {:ok, :started, {:continue, :ensure_all_started}}
          {:timeout, tables} ->
            Logger.error("Mnesia Unable to load tables: #{inspect tables} - shutting down...")
            :init.stop()
        end

      error -> :init.stop()
    end
  end

  def handle_continue(:ensure_all_started, state) do
    Logger.info("Starting server")
    {:ok, _started_apps} = :application.ensure_all_started(:server, :permanent)
    {:noreply, :started}
  end

  def copy_hex_cache(_, _, _) do
    File.cp_r!(Path.expand("~/.hex"), File.cwd!() <> "/hex")
    :ok
  end

  def delete_hex_cache(_, _, _) do
    File.rm_rf!(File.cwd!() <> "/hex")
    :ok
  end

end

Then it’s application file includes:

defmodule Bootstrap.Application do
  @moduledoc false

  use Application

  def start(_type, _args) do
    children = [
      {Bootstrap, []}
    ]

    opts = [strategy: :one_for_one, name: Bootstrap.Supervisor]
    Supervisor.start_link(children, opts)
  end
end

And it has a task in all similar to the phx.server task:

defmodule Mix.Tasks.Bootstrap do
  use Mix.Task

  def run(_) do
    Application.put_env(:phoenix, :serve_endpoints, true, persistent: true)
    {:ok, _} = Application.ensure_all_started(:bootstrap)
    Mix.Tasks.Run.run run_args() ++ ["--no-start"]
  end

  defp run_args do
    if iex_running?(), do: [], else: ["--no-halt"]
  end

  defp iex_running? do
    Code.ensure_loaded?(IEx) and IEx.started?
  end

end

This task is to be run when in dev, so that you can do iex -S mix bootstrap
In my case I do Ecto migrations when I need manually, but there’s nothing preventing you from adding a step to run migrations as well as part of the bootstrap gen_server. Any other things you need to do before starting your actual core, can be done here too if needed. When running the release it will start the bootstrap app (you don’t run the task) and that start your actual app.

I have found this a good way of structuring the startup of an application.

3 Likes
14

Thanks for the suggestions everyone! I looked at all of them and this reply as well: Change my mind: Migrations in a start phase.

I finally decided to use a Genserver like @axelson suggested, but ran it directly in my application supervision tree. Since I need to populate :persistent_term with DB rows, I need the Repo running. I could always start the repo process in another application that my main application depends on, but just running a temporary Genserver after the Repo process starts in the main application was simpler. Also, the Genserver's blocking init fn returns :ignore once it populates :persistent_term, so it exits normally without entering the msg receive loop. Works well for my use case :slight_smile:

2 Likes