How to test application startup?

Background

I want to test my Application module to see if is being launched correctly. To achieve this I am using Application.start in my tests, but it always returns {:error, {:already_started, _pid}}.

Test

To check if my app is launching correctly I have the following test:

defmodule FootbalInterface.ApplicationTest do
  use ExUnit.Case

  alias FootbalInterface.Application

  test "start code does not crash" do
    {:ok, _pid} = Application.start(:normal, [])
  end

  test "returns error when startup data is bad" do
    {:error, :invalid_startup_data} = Application.start(:normal, [])
  end
end

However this fails because it always returns {:error, {:already_started, _pid}}.
This happens because to run the tests mix launches my app first.

Question

How can I avoid this behaviour and write tests for my app without making mix launch it first?

mix test --no-start doesn’t start applications.

:wave:

Try testing the logic, not Application.start – move it into functions, and test them instead.

How do I use it with the tests of 1 module only?

So in order to test my startup, I have to make the private API of the Aplicationmodule public?
Is there any other solution?

By convention, exposing functions alone does not make them “public”, but only visible.

On the other hand, in my opinion, application start has to be treated axiomatic. It shall not do much anyway. Span the supervision. Rarely more.

Starting the application will work when everything else works.

How would you test the following start function then?

def start(_type, _args) do

    cowboy_pool =
      Cowboy.child_spec(
        scheme: :http,
        plug: Router,
        options: [port: 8080]
    )

    children = [
      cowboy_pool
    ]
    opts = [strategy: :one_for_one, name: MyApp.Supervisor]


    case Engine.start() do
      {:ok, :indexation_successful} ->
        Logger.info("Start successfull")
        Supervisor.start_link(children, opts)

      {:ok, :partial_indexation_successful, fails} ->
        Logger.warn("Partial start, limited functionality")
        Supervisor.start_link(children, opts)

      {:error, :no_valid_data_to_save} ->
        Logger.error("Unable to start engine, invalid data")
        {:error, :invalid_data}

      {:error, reason} ->
        Logger.error("Unable to start engine, unknown reason")
        {:error, reason}
    end

  end

I only start the supervisor, but I want to only start it when certain conditions are met.

I wouldn’t…

As I would build it in a way that there were no failable component…

Whatever Enige.start() does, should be part of the supervision tree, and defered in a way that your application doesn’t crash the beam, just because the function returned an error.

1 Like

I’d rewrite it as

  def start(_type, _args) do
    cowboy_plug =
      Cowboy.child_spec(
        scheme: :http,
        plug: Router,
        options: [port: 8080]
      )

    children = [
      cowboy_plug,
      Engine # need to add `start_link` to `Engine`
    ]
    
    opts = [strategy: :one_for_one, name: MyApp.Supervisor]
    Supervisor.start_link(children, opts)
  end

and test the engine startup logic in a separate engine_test.exs.

So in order to test my startup, I have to make the private API of the Aplication module public?

It won’t make you make any private functions public, since Engine.start (would need to add Engine.start_link as well) already seems to be public.

Engine.start() is a critical component of MyApp. Without it MyApp has no meaning, it’s like a car without its engine it’s not a car, it a carcass (ha, get it? car-cass? I’ll show myself out…).

Furthermore, this is a library, without processes nor supervision trees. It’s just a module with functions that don’t involve supervision trees. I can’t slap it on the children array for the supervisor to supervise because supervisor supervises processes and this wouldn’t fit.

I am struggling in understanding your solutions. Do you recommend I slap a process on this functional library just for the sake of not having to test the start function? I could really use an example with your way of doing this.

Well, as you called it Engine.start/0 I assumed this were backed by a process.

And as you said Engine is some library, that brings in some component that needs to be “started”, I’d make it an application with an empty supervision tree, to also ensure it can be terminated properly and clean up behind itself.

Then your application would depend on it, and since it can’t be properly started, BEAM wouldn’t even try to start your application.

Alternatively, your application wouldn’t make sense to do anything without that Engine started, so why not try to restart it? The remainder of your application could still serve some out of order notice instead of beeing completely unavailable.

1 Like

Then I’d call it Engine.init/0 to be more aligned with the current conventions.

Then you can add a task which starts the library into your supervision tree, it would exit once the engine inits.

    children = [
      cowboy_plug,
      {Task, fn -> Engine.init() end}
    ]
3 Likes

What happens if Engine.init fails?
That task fails silently and the app keeps running?

Or since we are placing an anon process in a Supervision Tree, will it try to restart the process X times until it gives up?

What happens if Engine.init fails?
That task fails silently and the app keeps running?

Depends on what Engine.init/0 does.
If it raises, the application startup would halt just as with your previous approach.

Just to make it sure, I understand this.

Your suggestions (from both @idiot and @NobbZ) is for me to turn the Engine library into an OTP application. The Enigine’s application would then have a Supervisor that would do the initialization. If this initialization fails, the Engine’s application dies.

So now I need MyApp (which depends on the Engine’s applciation) to put it into its Supervision tree. Ideally, there would be a Supervision strategy that simply keeps trying to launch failing children forever while the main app would still be up serving “I am not ready yet” responses.

I have a few issues with this approach. A few weeks ago I posted a discussion where I wanted a Supervisor to keep trying to run it’s children forever, as in, I didn’t want the Supervisor to ever crash, no matter how many times it’s children died. So I changed the default wait and retry values to absurd values, like 9999999.

This effectively meant that the Supervisor would always try to restart it’s children and the service would always be up.

The community, in general, disapproved of my solution because “if something is that wrong, better make sure it is visible by crashing loudly or simply stop retrying at all” (Summary from said discussion).

I am opting for the crash loudly part, but I understand I am being suggested to do exactly what I am not supposed to do - keep trying forever.

Could you help me understand where my logic is failing here?
My problem is that overall, I don’t understand how to make effective use of Supervision trees like you guys can.

You are mixing some proposals here…

Either you make it an application that supervises it self, if it crashes during its start, the BEAM won’t start further but instead crash before even reaching your application.

Or, you make it just something supervisable, that is hooked into your apps supervision tree. But restarts then shouldn’t be dealt with the usual OTP supervisions, but more specialised ones that are better suited for this task.

As far as I can remember, the specialised supervisor (eg some with exponential backoff) was the proposed solution in your other thread, rather than avoiding bubbling the crash…

Do you know of any resource you could recommend for me to follow?
Perhaps you know of a blog or book where this concept is more elaborated. I would really appreciate it.