Testing a Mix app with ExUnit: changing config values set at compile time

Questions / Help
, ,
5

I’ve finally had time to give it another go.

Thank you for your help @OvermindDL1, your solution worked really well.
I ended up using a Mix alias and also using different (unit tested) modules that are selected based on the configuration. The integration tests are then repeated with both configurations.

I’m going to summarize here the setup I’ve come up with, in case other people will find it helpful.
I’m leaving references to my specific use case in the code snippets to make it easier to follow their purpose.

###1) How to select different behavior at compile time

The cleanest solution I could find was to implement two modules with the same interface but with different implementations, so I ended up with FunWithFlags.Store that knows about both Redis and the ETS cache, and FunWithFlags.SimpleStore that only uses Redis directly. Both modules are unit tested independently.

Then, I select them at compile time with module attributes. For example:

  defmodule FunWithFlags.Config do
    def cache? do
      Application.get_env(:fun_with_flags, :cache, [enabled: true])[:enabled]
    end
+   def store_module do
+     if __MODULE__.cache? do
+       FunWithFlags.Store
+     else
+       FunWithFlags.SimpleStore
+     end
+   end
  end

  defmodule FunWithFlags do
-   alias FunWithFlags.Store
+   @store FunWithFlags.Config.store_module

    def do_something do
-     Store.do_something
+     @store.do_something
    end
  end

2) Define the configuration

While I provide a default value in the code, for this to work I used the default Mix configuration files. I’m only interested in the test environments, so I’m not bothering to load files for the dev env.

The main config file:

# config/config.exs

use Mix.Config

case Mix.env do
  :test          -> import_config "test.exs"
  :test_no_cache -> import_config "test_no_cache.exs"
  _              -> nil
end

The test one:

# config/test.exs

use Mix.Config

IO.puts "Loading config for default test env"

config :fun_with_flags, :cache,
  enabled: true

And the one that disables the cache:

# config/test_no_cache.exs

use Mix.Config

IO.puts "Loading config for the \"no cache\" integration test env"

config :fun_with_flags, :cache,
  enabled: false

3) Setup a Mix task alias

As documented here, it’s possible to define new Mix tasks that will be available only in this specific project without “bubbling up” to the host applications. I couldn’t really get a mix test override to work, but I actually realized that I find mix test.all more explicit and useful.

defmodule FunWithFlags.Mixfile do
  use Mix.Project

  def project do
    [
      aliases: aliases(),
      # ...
    ]
  end

  defp aliases do
    [
      {:"test.all", [&run_tests/1, &run_integration_tests/1]}
    ]
  end

  defp run_tests(_) do
    Mix.shell.cmd(
      "mix test --color", 
      env: [{"MIX_ENV", "test"}]
    )
  end

  defp run_integration_tests(_) do
    IO.puts "\nRunning the integration tests with the cache disabled."
    Mix.shell.cmd(
      "mix test test/fun_with_flags_test.exs --color",
      env: [{"MIX_ENV", "test_no_cache"}]
    )
  end
end

With this configuration, mix test is still available and the new alias acts as a simple wrapper. I also had to explicitly pass the --color flag because the output was using the default shell foreground color. Depending on what you’re used to, you might want to add more flags.

With this in place, I can run the new alias task and get the two test runs with both configurations. Mix will take care of recompiling the app for the second environment as appropriate.

$ rm -r _build
$ mix test.all
Loading config for default test env
==> connection
Compiling 1 file (.ex)
Generated connection app
==> redix
Compiling 9 files (.ex)
Generated redix app
==> fun_with_flags
Compiling 9 files (.ex)
Generated fun_with_flags app
Running the tests with Mix.env: test
...............................................

Finished in 0.1 seconds
47 tests, 0 failures

Randomized with seed 350185

Running the integration tests with the cache disabled.
Loading config for the "no cache" integration test env
==> connection
Compiling 1 file (.ex)
Generated connection app
==> redix
Compiling 9 files (.ex)
Generated redix app
==> fun_with_flags
Compiling 9 files (.ex)
Generated fun_with_flags app
Running the tests with Mix.env: test_no_cache
.........

Finished in 0.1 seconds
9 tests, 0 failures

Randomized with seed 153788

4) Setup Travis

The last bit is to use the same dual env system on Travis. This is as easy as configuring the script command in the travis.yml file:

  language: elixir
  elixir:
    - 1.4
  otp_release:
    - 19.2
  services:
    - redis-server
+ script: mix test.all
6 Likes