How do you run test in your umbrella app? When some apps want to use --no-start and some wants to normal behaviour

Chat / Discussions
, ,
1

Hi I am wondering how should I run test for my umbrella app.
given in the umbrella app there are -

  • apps that want to start supervision tree manually while testing- usually we use mix test --no-start in the individual app. See about this more here - https://virviil.github.io/2016/10/26/elixir-testing-without-starting-supervision-tree/. ( I am doing this due to, I want to setup and mock network calls and also test that what is sent on network is what I want during supervision loads).
  • apps that wants to start supervision tree - usually we use mix test
  • Some of my apps has ecto as it dependency - usually we alias `test: [ā€˜ecto.create --quiteā€™, ā€˜ecto.migrateā€™, test] in an individual app.
  • And one thing more, if two of our apps has ecto and it dependency how can we make sure that an app that has another app (letsā€™ called it app2, with ecto), the ecto of app2 is also migrated while running the test of the top level app.

ā€“ There are some constraint that I am facing -

  1. When I add alias [ā€œtestā€: [ā€œecto.createā€, ā€œecto.migrateā€, ā€œtestā€]] in the root of my umbrella app like what is recommend here https://github.com/elixir-ecto/ecto/issues/1538. This thing when get executed, it try to boot up the supervision trees of other apps also (which I donā€™t want it to be booted, since I want to use --no-start and I want to boot them manually)
  2. Currently to make sure that ecto migrations of dependency app is migrated, run ecto.setup(create and migrate db of all apps, This is added as an alias in root umbrella mix.exs,) before I start any test. (But by doing this here, it will force all apps supervision trees (since this ecto.migrate makes supervision tree to be loaded also), which is one thing I dont want because I wants to test some of my apps with --no-start.
1 Like
2

I am in a similar situation. Looking for a tutorial of some sorts on how to run ecto tests on oracle adapter I am writing!

1 Like
3

I believe adding a --no-start flag is not the right approach. What I Generally do is disable any modules that only add side-effect to the system.

What do I mean by this. IF you have a GenServer and itā€™s soul purpose is to automate the change of state of something over time. You should not start that particular GenServer in your application you should add the ability to disable Services on a per Server basis

Generally what I do is something like this in my GenServer.

  def start_link(_) do
    case GenServer.start_link(__MODULE__, nil, name: {:global, __MODULE__}) do
      {:ok, pid} ->
        {:ok, pid}

      {:error, {:already_started, pid}} ->
        Process.link(pid)
        {:ok, pid}

      :ignore ->
        :ignore
    end
  end

  def init(_) do
    if enabled() do
      fetch_currencies()
      {:ok, nil}
    else
      :ignore
    end
  end

  defp enabled do
    Application.get_env(:station, __MODULE__)[:enabled]
  end

After I add this now in my config/test.exs I can choose which service I want to disable.

config :station, Station.Currencies.Engine, enabled: false

I donā€™t need this GenServer to run because all it does is manipulate state over time. Which will cause all kinds of havoc in my tests. I want a deterministic test where the results donā€™t change over time.

Generally things like this is fine in production because you want results to change over time (for example currency exchange rate), if it goes up or down it will have an effect on the business accordingly but does not need to have an effect on my test. Iā€™m testing the validity of my system not how the change of data affects my bottom line.

So now I have the ability to start / disable GenServer / Services on a per module basis. I can selectively start the ones that are relevant for my test cases.

Which also generally means I will not put much logic in side said genserver. I will abstract the behavior into a module / function call and call it manually in my test setup as I need them. So fetch_currencies() call another function in my system which I can call myself at anytime.

4

I see what you meant there.

But by doing this, From what I understand we are not actually testing things that are written in Application Supervised GenServer cast or calls. And we are also assuming what is written in init function will have no effects in test environment. In general cases, I donā€™t think we can assume that.

This is what I think, In order to test init, and handle_cast, handle_call, handle_info calls of these Application supervised genservers, we should manually start application (spawn the application supervisor and test things).

These tests that I am referring to is the result of execution of init, handle_cast, handle_call and handle_info calls.

1 Like
5

You can. You can start it up individually in the different test cases. All you have to do is call start_link etc yourself get the pid and pass it into the test context which you can then call what you need to he genserver.

Also the enabled() function can be customized, you can pass an option that overrides the Application config just for that test case.

6

Hi @zacksiri, @c-bik

I think I found the best way to do tests without using --no-start parameters and without using enable() function to check if I should enable that service in the test or not. By using this method it will guarantee that for each test, the application is always restarted.

The way to do this is pretty simple, We just need to make the ex_unit runs in synchronous mode and then we add code in to the testsā€™ setup block that stop the application and start the application again.

For example the code could look like below -

defmodule MyApp.ModuleTest do
  use ExUnit, async: false

  setup do
    Application.stop(:my_app)

    :ok = Application.start(:my_app)
  end 
  .
  .
  .
end

The answer to this could be found in -


and

I think there is one more question that I would like to ask for community best practice, is how to handle, the case where some of dependency apps (apps that are added inside other mix.exs app in umbrella app) also has ecto migrations to be migrated or seeds file to be seeded before the app test can be run. In my personal thinking, I just donā€™t think that running ecto migration on the root level of umbrella app before running test is the right way to do.

One of the reason why I think that I should not run migration and seeds on top level is that, and an app should not have dependency that maintain non temporal states(in here non temporal states mean the states that are persisted, or states that can not be discarded after some actions. Some of the actions could be running a test)
Generally when we test app with ecto, we usually configure ecto to run in sandbox for that app. but when this app is having dependencies of another apps that have ecto also, how can this app controls or has some knowledge of whether the sub apps that have ecto are running correctly and are running in the right mode and how can we be sure that after a test (test that contains creating and deleting records on the subapps)has finished running, the sub apps should discarded those changes after each test finishes)

1 Like