Is there a way to set ExUnit context from the CLI?

Questions / Help
1

Is there a way to modify the ExUnit test context from the CLI? E.g. instead of explicitly doing something like

@tag foo: "bar"
test "something", %{foo: foo} do
   # ...

I’m wondering if it’s possible to do this via the command line, e.g. mix test --foo bar
I haven’t spotted anything in the docs, however.

The closest thing I’ve come up with is to us set an ENV var that affects config, e.g. FOO=bar mix test. That’s arguably not as elegant.

Thanks for a sanity check!

2

I’m pretty sure there is no way other than using ENV vars as you said. Though depending on what you are trying to do, some combination of the --only and --exclude options could work so you can run different tests in different situations.

1 Like
3

Can you elaborate a bit more on your use case? One major difference between tags / context is that tags are actual elixir values, and command line arguments are all going to be strings.

4

The use case is admittedly a bit weird, but I wanted to field a couple alternatives for people on our team who may want to organize code differently. The thing that lead to this question deals with making requests to an external API. For regular unit tests, we use Mox to create a mock HTTP client and use that to make assertions and avoid making live HTTP requests.

Things get more interesting when we tried to field the desire to do some REAL requests in our tests (i.e. more integration tests). Broadly, the 2 solutions cooked up in our thinktank were these:

Option 1. Use tags in the tests to unequivocally designate which tests made real HTTP requests (and those tests would be skipped by default). e.g.

describe "real stuff" do
  @describetag :external
  test "some endpoint" do
    MyAPI.make_request("foo", client: RealHTTPClient)
  end
end

describe "mock stuff" do
  test "some endpoint" do
    MyAPI.make_request("foo", client: MockHTTPClient)
  end
end

Option 2. Use configuration to swap out the HTTP client implementation. The pseudo-code detailing this switch-a-roo looks something like this:

defp http_client do
  if System.get_env("LIVE") do
    RealHTTPClient
  else
    MockHTTPClient
  end
end

test "some endpoint" do
  client = http_client()

  MyAPI.make_request("foo", client: client)
end

I think Option 1 is more idiomatic, but I can see some pros to Option 2.

5

There is no “clean” way to do it; tag inputs are mostly meant for filtering the tests you run, i.e.

@tag :frontend

So you could run just the frontend tests via mix test --only frontend or my favorite mix test --only wip

The only real way to achieve it is by envars, and before you do that you should really evaluate if thats something you want to do. It makes your tests a bit more brittle, and subject to external factors. Better to write a test suite that tests all possible values of the variable, such as using a for loop to iterate over them, tagging the different variations with values, and then filtering at runtime using the --only param

1 Like