Separate mix commands for unit and integration tests

[Fl4m3Ph03n1x]

Background

I have a project where I have separated my tests into two categories: Unit tests and Integration tests.

My objective is to have 3 commands that do the following:

• mix test.unit: executes all unit tests
• mix test.integration: executes all integration tests
• mix test: executes mix test.unit and then mix test.integration

I can somewhat replicate my first objective, instead of mix test.unit I am using environment variables and I am running MIX_ENV=unit mix test.

Problem

There are however a few problems with this approach:

• It is way more verbose than the intended command
• The only way I know of taking advantage of this is by littering my mix.exs with environment variables and pattern matching function that do something depending on the environment
• It is very easy to miss the environment variable name only to see things blow up
• I dont get help not auto complete if I type mix help

Research

I thought about using a mix alias like in this post:

How do I set the env at alias??

Hi all, I am a newbie to Elixir. I try to create an alias for running another group of tests which has its owned environment. It can be run with MIX_ENV specified. May I set the env at alias to have a simpler command? Like api_test: defp aliases do [ "ecto.setup": ["ecto.create", "ecto.migrate", "run priv/repo/seeds.exs"], "ecto.reset": ["ecto.drop", "ecto.setup"], "test": ["ecto.create --quiet", "ecto.migrate", "test"], "api_test": "test --only api_test:true" ] end

But not only am I unsure on how to do it, I don’t even think I can change the default behavior of mix test to run both unit and integration tests separately.

Code

Thus far, all I have no is a half-broken mix.exs file that works:

defmodule MarketManager.MixProject do
  use Mix.Project

  @test_envs [:unit, :integration]

  ##########
  # Public #
  ##########

  def project do
    [
      app: :market_manager,
      version: "0.1.0",
      elixir: "~> 1.9",
      start_permanent: Mix.env() == :prod,
      deps: deps(),
      elixirc_paths: elixirc_paths(Mix.env),
      test_paths: test_paths(Mix.env)
    ]
  end

  # Run "mix help compile.app" to learn about applications.
  def application do
    [
      extra_applications: applications(Mix.env),
      mod: {MarketManager.Application, [env: Mix.env]}
    ]
  end

  ###########
  # Private #
  ###########

  defp applications(:integration), do: applications(:default) ++ [:cowboy, :plug]
  defp applications(_),     do: [:logger]

  # Run "mix help deps" to learn about dependencies.
  defp deps do
    [
      {:httpoison, "~> 1.6"},
      {:jason, "~> 1.2"},

      # Testing and Dev
      {:hammox, "~> 0.2", only: @test_envs},
      {:mix_test_watch, "~> 1.0", only: @test_envs, runtime: false},
      {:plug_cowboy, "~> 2.0", only: @test_envs}
    ]
  end

  defp elixirc_paths(env) when env in @test_envs, do: ["test/support", "lib"]
  defp elixirc_paths(_),     do: ["lib"]

  defp test_paths(:integration), do: ["test/integration"]
  defp test_paths(:unit), do: ["test/unit"]
  defp test_paths(_), do: ["test/unit"]

end

Which was inspired in the blog:

Testing in Elixir with Multiple Environments

Questions

As you can probably tell from reading my mix.exs file, it is a half broken mess where, in order to compile, I have to mix the definitions of MIX_ENV=unit mix test and mix test (check the test_paths) functions.

This brings all sorts of pain when developing.

How can I fix this file, so it has the desired behavior?

[shanesveller]

Consider using @tag and @moduletag syntax in your test modules together with --include and/or --exclude flags to the existing `mix test’ task. This could then be automated as new mix aliases. IIRC your goal is a canonical example of why those flags exist.

[LostKobrakai]

I’m wondering why you need :extra_applications here? Having the dependencies marked for only the :test env should be enough.

[Fl4m3Ph03n1x]

The issue I see with that approach is that I have to add an extra @moduletag to every module, while if I manage to do it my way, all I have to do is put the test files inside the “integration” and “unit” folders and it will be self explanatory.

This said, that is a valid approach. Depending on how much trouble the setup I am looking for takes, I may consider your approach.

[cpgo]

I dont know if it works but maybe you could try to create a base case with the moduletags for each test type
https://hexdocs.pm/ex_unit/ExUnit.Case.html

[LostKobrakai]

@Fl4m3Ph03n1x

I have to add an extra @moduletag to every module

If you have two types of tests it’s enough to tag the one with the smaller numbers and either run everything except that one tag or nothing, but that tag.

[Fl4m3Ph03n1x]

This is from a blog I saw:

Testing External Web Requests in Elixir? Roll Your Own Mock Server

# mix.exs
def application do
  [
    extra_applications: [:logger],
    mod: {GithubClient.Application, [env: Mix.env]},
    applications: applications(Mix.env)
  ]
end

I think the author made a little confusion because she uses both applications and extra_applications and one of them was discontinued iirc, but overall it has valuable information.