ExUnit tests from a module instead of command line/mix

redrapids · May 16, 2019, 3:46pm

Is there a way to run ExUnit tests programmatically? If so is there an example somewhere that tells me how?

-bt

rodrigues · May 16, 2019, 3:50pm

Maybe ExUnit.run() ? https://hexdocs.pm/ex_unit/ExUnit.html#run/0

EDIT: didn’t manage to run that locally yet:

$ MIX_ENV=test iex -S mix
Erlang/OTP 21 [erts-10.3.5] [source] [64-bit] [smp:4:4] [ds:4:4:10] [async-threads:1] [hipe] [dtrace]

Interactive Elixir (1.8.2) - press Ctrl+C to exit (type h() ENTER for help)
iex(1)> ExUnit.start(autorun: false)
:ok
iex(2)> result = ExUnit.run()
** (exit) exited in: GenServer.call(ExUnit.Server, {:take_async_modules, 8}, 60000)
    ** (EXIT) time out
    (elixir) lib/gen_server.ex:989: GenServer.call/3
    (ex_unit) lib/ex_unit/runner.ex:70: ExUnit.Runner.loop/3
    (stdlib) timer.erl:166: :timer.tc/1
    (ex_unit) lib/ex_unit/runner.ex:17: ExUnit.Runner.run/2

hauleth · May 16, 2019, 4:26pm

Of course it is, as @rodrigues said you need to use ExUnit.run/0, but this is a little bit more complicated because there are few steps needed before:

You need to start ex_unit application via Application.ensure_all_started(:ex_unit)
You need to compile and load all test modules
You need to configure ExUnit to not run on exit

In general if you want to study how to run it programmatically then check out mix test task code.

PS I believe it should be done simpler in the way that you can run Common Test via simple command ct:run([]) in Erlang.

redrapids · May 16, 2019, 4:39pm

Yeah… that’s where I am now… was thinking there might be a more focused example somewhere.

I might switch to test configuration and formatters to get the behavior I want.

Thanks for your help all.

-bt

hauleth · May 16, 2019, 4:51pm

Main question is what you want to achieve, otherwise it is very easy to land on XY problem, so looking for solution for wrong question.

rodrigues · May 16, 2019, 5:04pm

That’s how I could call it from iex:

➜ env MIX_ENV=test iex -S mix
Erlang/OTP 21 [erts-10.3.5] [source] [64-bit] [smp:4:4] [ds:4:4:10] [async-threads:1] [hipe] [dtrace]

Interactive Elixir (1.8.2) - press Ctrl+C to exit (type h() ENTER for help)
iex(1)> ExUnit.start(autorun: false)
:ok
iex(2)> test_files = Path.wildcard("test/**/*_test.exs")
["test/lib/..._test.exs", ...]
iex(3)> Mix.Compilers.Test.require_and_run(test_files, ["test"], formatters: [ExUnit.CLIFormatter])
...................................................................................................................................................................................................................................................

Finished in 69.6 seconds
193 properties, 50 tests, 0 failures

Randomized with seed 791939
{:ok, %{excluded: 0, failures: 0, skipped: 0, total: 243}}
iex(5)>

gregvaughn · May 16, 2019, 5:18pm

I’m not certain this is quite what you want, but this is how I did some of my early learning exercises in Elixir. Execute the single file with elixir roman_numerals.exs

github.com

gvaughn/elixir_kata/blob/master/roman/roman_numerals.exs

defmodule RomanNumerals do

  @moduledoc """
  Task definition: Create a function taking a positive integer as its parameter and
  returning a string containing the Roman Numeral representation of that integer.
  """

  @mappings [{"X", 10}, {"IX", 9}, {"V", 5}, {"IV", 4}, {"I", 1}]

  def convert(number) do
    {0, str} = Enum.reduce(@mappings, {number, ""},
      fn ({roman, arabic}, {number, str}) ->
          {rem(number, arabic), str <> String.duplicate(roman, div(number, arabic)) }
      end)
    str
  end
end

ExUnit.start

This file has been truncated. show original

redrapids · May 16, 2019, 6:27pm

Thanks all!

I have plenty to play with now.

-bt

axelson · September 13, 2019, 8:14pm

Did you ever get something working for this?

neenjaw · May 9, 2021, 10:34pm

Bumping again 2 years later, also curious if you got something working. I’ve been able to successfully trigger the tests, but on a repeated run, it cache’s the previous test run’s result and returns as though no tests were run? Is there a way to reset that cache?

thbar · May 12, 2021, 11:54am

I am also interested. The reason of my interest is the use of unit-testing frameworks to write what I call “data screens”, which are data audit verifications that one can run on a live system. This can be used for data migrations but also all types of ETL / data-pipelines works.

I would love to able to run ExUnit suites in fully independent fashion, with a formatted output via a reporter, without global state shared between suites (so that they could be run as part of background Oban jobs or similar).

A trick I’ve used in the past (with Ruby) is leveraging only the assertions part:

class Wrapper
  include MiniTest::Assertions
  attr_accessor :assertions

  def initialize
    # https://github.com/seattlerb/minitest/issues/286
    self.assertions = 0
  end
end

It could be a good first step to only rely on ExUnit.Assertions similarly (ExUnit.Assertions — ExUnit v1.11.4), at the expense of having to write your own “runner”.

If anyone has more ideas (or feedback from @redrapids), I’m interested!