ExUnit tests from a module instead of command line/mix

Is there a way to run ExUnit tests programmatically? If so is there an example somewhere that tells me how?

-bt

2 Likes

Maybe ExUnit.run() ? https://hexdocs.pm/ex_unit/ExUnit.html#run/0

EDIT: didn’t manage to run that locally yet:

$ MIX_ENV=test iex -S mix
Erlang/OTP 21 [erts-10.3.5] [source] [64-bit] [smp:4:4] [ds:4:4:10] [async-threads:1] [hipe] [dtrace]

Interactive Elixir (1.8.2) - press Ctrl+C to exit (type h() ENTER for help)
iex(1)> ExUnit.start(autorun: false)
:ok
iex(2)> result = ExUnit.run()
** (exit) exited in: GenServer.call(ExUnit.Server, {:take_async_modules, 8}, 60000)
    ** (EXIT) time out
    (elixir) lib/gen_server.ex:989: GenServer.call/3
    (ex_unit) lib/ex_unit/runner.ex:70: ExUnit.Runner.loop/3
    (stdlib) timer.erl:166: :timer.tc/1
    (ex_unit) lib/ex_unit/runner.ex:17: ExUnit.Runner.run/2
1 Like

Of course it is, as @rodrigues said you need to use ExUnit.run/0, but this is a little bit more complicated because there are few steps needed before:

  • You need to start ex_unit application via Application.ensure_all_started(:ex_unit)
  • You need to compile and load all test modules
  • You need to configure ExUnit to not run on exit

In general if you want to study how to run it programmatically then check out mix test task code.


PS I believe it should be done simpler in the way that you can run Common Test via simple command ct:run([]) in Erlang.

3 Likes

Yeah… that’s where I am now… was thinking there might be a more focused example somewhere.

I might switch to test configuration and formatters to get the behavior I want.

Thanks for your help all.

-bt

Main question is what you want to achieve, otherwise it is very easy to land on XY problem, so looking for solution for wrong question.

That’s how I could call it from iex:

➜ env MIX_ENV=test iex -S mix
Erlang/OTP 21 [erts-10.3.5] [source] [64-bit] [smp:4:4] [ds:4:4:10] [async-threads:1] [hipe] [dtrace]

Interactive Elixir (1.8.2) - press Ctrl+C to exit (type h() ENTER for help)
iex(1)> ExUnit.start(autorun: false)
:ok
iex(2)> test_files = Path.wildcard("test/**/*_test.exs")
["test/lib/..._test.exs", ...]
iex(3)> Mix.Compilers.Test.require_and_run(test_files, ["test"], formatters: [ExUnit.CLIFormatter])
...................................................................................................................................................................................................................................................

Finished in 69.6 seconds
193 properties, 50 tests, 0 failures

Randomized with seed 791939
{:ok, %{excluded: 0, failures: 0, skipped: 0, total: 243}}
iex(5)>
4 Likes

I’m not certain this is quite what you want, but this is how I did some of my early learning exercises in Elixir. Execute the single file with elixir roman_numerals.exs

2 Likes

Thanks all!

I have plenty to play with now.

-bt

Did you ever get something working for this?

1 Like

Bumping again 2 years later, also curious if you got something working. I’ve been able to successfully trigger the tests, but on a repeated run, it cache’s the previous test run’s result and returns as though no tests were run? Is there a way to reset that cache?

I am also interested. The reason of my interest is the use of unit-testing frameworks to write what I call “data screens”, which are data audit verifications that one can run on a live system. This can be used for data migrations but also all types of ETL / data-pipelines works.

I would love to able to run ExUnit suites in fully independent fashion, with a formatted output via a reporter, without global state shared between suites (so that they could be run as part of background Oban jobs or similar).

A trick I’ve used in the past (with Ruby) is leveraging only the assertions part:

class Wrapper
  include MiniTest::Assertions
  attr_accessor :assertions

  def initialize
    # https://github.com/seattlerb/minitest/issues/286
    self.assertions = 0
  end
end

It could be a good first step to only rely on ExUnit.Assertions similarly (ExUnit.Assertions — ExUnit v1.11.4), at the expense of having to write your own “runner”.

If anyone has more ideas (or feedback from @redrapids), I’m interested!