How to start a GenServer-based Application before other Applications?

Hey there!

This probably has a very easy answer, but I’m already too many hours into this and need some help. In this scenario, we have several separated git repositories, one being a library to provide configuration values and the others are applications that use the mentioned lib.

The library used to be a plain stateless module and this has been working well for years. Recently, we had to change this library to include some caching and different data sources, which caused a GenServer to be added. It means that it ceased to be a simple bunch of functions and now requires to be fully loaded and started to be used.

One peculiarity, being a config provider, this library is usually invoked from start/2 callbacks of the other Applications, hence it has to be up and running before the other applications have their start/2 functions called.

The (greatly simplified) structure is as follows:

Library: mix.exs

defmodule Foo.Config.MixProject do
  use Mix.Project

  def project do
    [
      app: :foo_config,
      version: "2.0.0",
      elixir: "~> 1.10",
      deps: [{:jason, "~> 1.2"}],
    ]
  end

  def application do
    [
      extra_applications: [:logger],
      mods: {Foo.Config.Application, []},
    ]
  end
end

Library: lib/application.ex

defmodule Foo.Config.Application do
  @moduledoc false
  use Application

  @impl true
  def start(_type, _args) do
    children = [Foo.Config.Server]
    opts = [strategy: :one_for_one, name: Foo.Config.Supervisor]
    Supervisor.start_link children, opts
  end
end

Library: lib/config.ex

defmodule Foo.Config do
  alias Foo.Config.Server

  def get_config(key_path) when is_binary(key_path) do
    GenServer.call Server.name, {:get_config, key_path}
  end
end

Library: lib/server.ex

defmodule Foo.Config.Server do
  use GenServer
  require Logger

  @server_name {:global, __MODULE__}

  def name, do: @server_name

  def start_link(_opts) do
    GenServer.start_link __MODULE__, [], name: @server_name
  end

  def child_spec(_args) do
    %{
      id: Foo.Config.Server,
      start: {Foo.Config.Server, :start_link, [nil]},
    }
  end

  @impl GenServer
  def init(_opts) do
    # Some operations to load initial state, potentially expensive
    {:ok, %{config: initial_state}}
  end

  @impl GenServer
  def handle_call({:get_config, key_path}, _from, %{config: data} = state)
    when is_binary(key_path) and is_map(data)
  do
    # Uses private functions to calculate the value to be returned
    {:reply, value, state}
  end
end

Application: mix.exs

defmodule Bar.MixProject do
  use Mix.Project

  def project do
    [
      app: :bar,
      version: "2.1.0",
      elixir: "~> 1.10",
      start_permanent: Mix.env() == :prod,
      deps: deps(),
      test_paths: test_paths(Mix.env),
      releases: [
        bar: [
          include_executables_for: [:unix]
        ]
      ]
    ]
  end

  def application do
    [
      extra_applications: [
        :logger,
        # We tried to add the library here, but didn't help
        :foo_config,
      ],
      mod: mods(Mix.env),
    ]
  end

  defp mods(:unit), do: []
  defp mods(_all), do: {Bar.Application, []}

  defp test_paths(:integration), do: ["test/integration"]
  defp test_paths(:unit), do: ["test/unit"]
  defp test_paths(_), do: []

  defp deps do
    [
      {:argon2_elixir, "~> 2.3"},
      {:joken, "~> 2.2"},
      {:mongodb_driver, "~> 0.7"},
      {:plug_cowboy, "~> 2.2"},
      {
        :foo_config,
        git: "git@github.com:my-company/foo-config.git",
        tag: "v2.0.0",
      },
    ]
  end
end

Application: lib/bar/application.ex

defmodule Bar.Application do
  use Application

  # We tried to use `require` here, but no effect
  require Foo.Config

  import Foo.Config, only: [
    get_config: 1,
    get_secret: 1,
  ]

  require Logger

  def start(_type, _args) do
    # We also tried the lines here, but no luck:
    # {:ok, _} = Application.ensure_all_started :foo_config
    # Process.sleep 5000

    {:ok, port} = get_config "port"
    {:ok, db_url} = get_config "uri"
    {:ok, db_user} = get_config "username"
    {:ok, db_pass} = get_config "password"

    children = [
      {Plug.Cowboy, [
        scheme: :http,
        plug: Bar.Router,
        options: [ip: {0, 0, 0, 0}, port: port],
      ]},
      {Mongo, [
        name: :db,
        appname: "Bar",
        url: db_url,
        username: db_user,
        password: db_pass,
        # More config here, obtained through `get_config/1`
      ]},
    ]

    opts = [name: Bar.Supervisor, strategy: :one_for_one]
    Logger.info "Starting Bar.Supervisor"
    Supervisor.start_link children, opts
  end
end

Application: many other files, such as router.ex

defmodule Foo.Router do
  use Plug.Router
  # Lots of magic here
end

Compilation runs just fine, but trying to run the application always results in the following error (extracted from docker-compose logs). The CMD for this container during development is mix do deps.get, run --no-halt and bar start for the release in production:

app_1          | ** (Mix) Could not start application bar: exited in: Bar.Application.start(:normal, [])                                                              
app_1          |     ** (EXIT) exited in: GenServer.call({:global, Foo.Config.Server}, {:get_config, "port"}, 5000)                                                            
app_1          |         ** (EXIT) no process: the process is not alive or there's no process currently associated with the given name, possibly because its application isn't started

What am I missing here? Suggestions about project structure are also welcome, I just really need to keep the config provider as an independent library because many different applications (deployed as independent Erlang nodes and containers) are using the same logic.

Thanks in advance! Live long and prosper \\//_

I recommend to keep it as a plain stateless module, and start one genserver for each application that need the configuration service, as part of their own supervision trees. The genserver code is still in your library; all you need to provide is a childspec function that take a name key.

If you still prefer to have a stateful application and can live with the fact that it is a singleton, you can just make sure each application that use its service listing the library as a dependency. Depended application always start earlier than the dependent. There is no need to use extra_applications

The idea of having the GenServer code there, avoid the singleton and start it under the supervision tree of the application is interesting. I can try to apply this refactoring suggestion, but the question remains on how to make sure to start this GenServer before the other processes, so they can call get_config/1 during their own start/2 functions.

About the second idea, the singleton design here is actually desirable, because certain configuration keys require reaching out for external sources that cost money on every single request (that’s a nice side-effect, isn’t it?). Using a singleton minimizes such calls and the overall costs of the application. We tried the approach you described, by just adding it as a dependency, but unfortunately it didn’t work.

The mix compile.app docs mention a mod: (singular!) option, not a mods:.

1 Like

Unbelievable… That was it! A stupid typo, everything works… :man_facepalming:

@al2o3cr if you are in Berlin, the beer and/or coffee is on me! Thanks for saving my sanity.