Managing different test environments with mix aliases

Hi. My project has both unit and property tests. Unit tests are using a mocking library but I don’t need my property tests to run with the mocks, so I decided to split my test env into two: test and property_test using test.exs and property_test.exs config files respectively. Finally, I’m running these tests using the commands:

$ MIX_ENV=property_test  mix test  test/property/
$ MIX_ENV=test mix test test/unit

Now, I would like to instead create custom commands to simplify things using Mix’s aliases but I don’t know how to achieve it. Like how would that look like ? I don’t know how to tell Mix which env it should execute each alias with. This is what I have so far in my mix.exs

  def cli do
    [preferred_envs: ["test.unit": :test, "test.property": :property_test]]
  end

  defp aliases() do
  [
      "test.property": ["test test/property/"],
      "test.unit": ["test test/unit"]
    ]
  end

But I’m still getting this error

** (Mix) “mix test” is running in the “property_test” environment. If you are running tests from within another command, you can either:

  1. set MIX_ENV explicitly:

    MIX_ENV=test mix test.another

  2. set the :preferred_envs for “def cli” in your mix.exs:

    def cli do
    [preferred_envs: [“test.another”: :test]]
    end

Ok, so I actually don’t know why this approach fails, but I managed to make it work with this hack in the aliases definition (everything else needs to stay the same).

  defp aliases() do
    [
      "test.property": [
        fn _ -> System.put_env("MIX_ENV", "property_test") end,
        "test test/property"
      ],
      "test.unit": ["test test/unit"]
    ]
  end

Forciby setting the MIX_ENV variable plus setting the preferred envs in the cli function does it. This seems like a bug to me, I’m using Elixir 1.16.0 if it helps anyone else.

I feel like you went really deep for something that should be as trivial as “don’t opt into mocks inside your property tests”. Was that attempted?

I don’t know how to tell my property tests to not use mocks.

I basically have this at the top of the file I want to test:

def external_api, do: Application.get_env(:my_app, :external_api)

...
def fetch_data do
   data = external_api().fetch_data()
  ... 
end

..

So whenever I’m on test mode I get a mock, and I get the real API module in any other mode . Because both property and unit tests run on the test environment by default, I couldn’t figure out a way to separate them other than the approached mentioned in my post.

Sure, my point was that just for one or more tests you can skip the external_api() mediator and use the real thing instead.