How to test macros?

Questions / Help
,
1

Is it possible to get some test coverage for custom macros?
Is ExUnit able to test different compilation scenarios?

$ mix test --cover  

  0.00% | MyApp.MyMacro

Online searches result in stories on how ExUnit is implemented using macros and the various macros ExUnit has on offer.

2

Coverage on the beam is calculated using erlangs :cover module, which is unaware of macros.

3

Thx for clearing that up :+1:

4

That said, it will do coverage on the code generated and inserted in any function calling a macro so I suppose that would be some kind of coverage?

5

Actually all of the modules using this particular macro have a 100% coverage.
That should give me at least partial coverage then right?

6

Never mind, that’s silly.

7

I found it is possible to get pretty good test coverage (near 100%) using __after_compile__/2 in combination with assert_raise/2 in tests.

test "some aspect of my macro" do
  assert_raise SomeError, fn ->
    defmodule Dummy do
      use MyMacro, option: "testable"
    end
  end
end

Make options available (for example using a module attribute) to be able to validate after compilation.

defmacro __using__(opts) do
  option = Keyword.fetch!(opts, :option)
  quote do
    @__option__ unquote(option)
    @after_compile unquote(__MODULE__)
    # implementation
  end
end

def __after_compile__(env, _bytecode) do
  option = Module.get_attribute(env.module, :__option__)
  # validate option
  # maybe raise exception, assert_raise in test
  # when option is a capture, run it.
end
8

Here you can find how I managed testing macros.

@LostKobrakai while macros are “special functions” these are still functions and with some tricks (see above) you can test them and get coverage for them without problems.

9

That’s very interesting.

10

Kind of the same topic:

11

In “Metaprogramming Elixir: Write Less Code, Get More Done (and Have Fun!)” Chris McCord (author of Phoenix) writes that tests should best assert on the desired behavior the macro tries to achieve and not the particular code generated by it (or even the AST).

In general I can recommend the book, it’s an enlightening and surprisingly short read.

12

Though I agree for the most part I do like to make sure that for example passing invalid options to macros will crash the program and getting better feedback fast is also a plus for me.
In general I (try to) run a tight ship, more so than most Elixir Gurus from what I can tell.