Separate mix commands for unit and integration tests


I have a project where I have separated my tests into two categories: Unit tests and Integration tests.

My objective is to have 3 commands that do the following:

  • mix test.unit: executes all unit tests
  • mix test.integration: executes all integration tests
  • mix test: executes mix test.unit and then mix test.integration

I can somewhat replicate my first objective, instead of mix test.unit I am using environment variables and I am running MIX_ENV=unit mix test.


There are however a few problems with this approach:

  • It is way more verbose than the intended command
  • The only way I know of taking advantage of this is by littering my mix.exs with environment variables and pattern matching function that do something depending on the environment
  • It is very easy to miss the environment variable name only to see things blow up
  • I dont get help not auto complete if I type mix help


I thought about using a mix alias like in this post:

But not only am I unsure on how to do it, I don’t even think I can change the default behavior of mix test to run both unit and integration tests separately.


Thus far, all I have no is a half-broken mix.exs file that works:

defmodule MarketManager.MixProject do
  use Mix.Project

  @test_envs [:unit, :integration]

  # Public #

  def project do
      app: :market_manager,
      version: "0.1.0",
      elixir: "~> 1.9",
      start_permanent: Mix.env() == :prod,
      deps: deps(),
      elixirc_paths: elixirc_paths(Mix.env),
      test_paths: test_paths(Mix.env)

  # Run "mix help" to learn about applications.
  def application do
      extra_applications: applications(Mix.env),
      mod: {MarketManager.Application, [env: Mix.env]}

  # Private #

  defp applications(:integration), do: applications(:default) ++ [:cowboy, :plug]
  defp applications(_),     do: [:logger]

  # Run "mix help deps" to learn about dependencies.
  defp deps do
      {:httpoison, "~> 1.6"},
      {:jason, "~> 1.2"},

      # Testing and Dev
      {:hammox, "~> 0.2", only: @test_envs},
      {:mix_test_watch, "~> 1.0", only: @test_envs, runtime: false},
      {:plug_cowboy, "~> 2.0", only: @test_envs}

  defp elixirc_paths(env) when env in @test_envs, do: ["test/support", "lib"]
  defp elixirc_paths(_),     do: ["lib"]

  defp test_paths(:integration), do: ["test/integration"]
  defp test_paths(:unit), do: ["test/unit"]
  defp test_paths(_), do: ["test/unit"]


Which was inspired in the blog:


As you can probably tell from reading my mix.exs file, it is a half broken mess where, in order to compile, I have to mix the definitions of MIX_ENV=unit mix test and mix test (check the test_paths) functions.

This brings all sorts of pain when developing.

How can I fix this file, so it has the desired behavior?

Consider using @tag and @moduletag syntax in your test modules together with --include and/or --exclude flags to the existing `mix test’ task. This could then be automated as new mix aliases. IIRC your goal is a canonical example of why those flags exist.

I’m wondering why you need :extra_applications here? Having the dependencies marked for only the :test env should be enough.

The issue I see with that approach is that I have to add an extra @moduletag to every module, while if I manage to do it my way, all I have to do is put the test files inside the “integration” and “unit” folders and it will be self explanatory.

This said, that is a valid approach. Depending on how much trouble the setup I am looking for takes, I may consider your approach.

1 Like

I dont know if it works but maybe you could try to create a base case with the moduletags for each test type

If you have two types of tests it’s enough to tag the one with the smaller numbers and either run everything except that one tag or nothing, but that tag.

This is from a blog I saw:

# mix.exs
def application do
    extra_applications: [:logger],
    mod: {GithubClient.Application, [env: Mix.env]},
    applications: applications(Mix.env)

I think the author made a little confusion because she uses both applications and extra_applications and one of them was discontinued iirc, but overall it has valuable information.