Capture compiler warnings in tests

grych · February 16, 2018, 11:45am

Is it possible to capture compiler warnings in tests? I am doing a following test which should raise CompileError (and it does, test is successful):

@tag capture_log: true
test "update child should raise", fixture do
  IO.puts("\n--> the following warning is expected:")
  assert_raise CompileError, fn -> poke(fixture.socket, excluded: "Hegemon") end
end

But I am getting this warning during the tests:

warning: variable "u" does not exist and is being expanded to "u()", please use parentheses to remove the ambiguity or change the variable name
  nofile:6

This behaviour is expected. Indeed, u is not defined in the AST I am processing. Is there a way I can capture this? capture_log did not help.

OvermindDL1 · February 16, 2018, 4:25pm

I use code like this in my macro-heavy projects:

github.com

OvermindDL1/typed_elixir/blob/master/test/test_helper.exs

ExUnit.start()
ExUnit.configure(colors: [enabled: true])

defmodule CompileTimeAssertions do
  defmodule DidNotRaise, do: defstruct(message: nil)

  defmacro assert_compile_time_raise(expected_exception, expected_message, fun) do
    actual_exception =
      try do
        Code.eval_quoted(fun)
        %DidNotRaise{}
      rescue
        e -> e
      end

    quote do
      assert unquote(actual_exception.__struct__) === unquote(expected_exception)
      assert unquote(actual_exception.message) === unquote(expected_message)
    end
  end

This file has been truncated. show original

And it’s used like this:

github.com

OvermindDL1/typed_elixir/blob/master/test/typed_elixir_test.exs#L23-L29


assert_compile_time_throw {:NO_TYPE_RESOLUTION, %TypedElixir.Type.Const{const: :integer, meta: %{values: [1]}}, %TypedElixir.Type.Const{const: :atom, meta: %{values: [nil]}}}, fn ->
  use TypedElixir
    defmodulet TypedTest_Typed_Simple do
      @spec simple() :: nil
      def simple(), do: 1
    end
end

I probably should pull my test helpers into their own library sometime. They are based on someone else’s but expanded for more features (and a bug fix).

grych · February 16, 2018, 4:42pm

Very interesting, thanks for sharing!

Anyway, I think I can’t use it in my case. The whole stuff is going on in runtime (I am evaluating previously saved AST). Also, assert_raise is working correctly in this case, function raises CompileError as expected. The only small issue is this warning, which I can live with, but it is annoying

OvermindDL1 · February 16, 2018, 4:44pm

In that case I’d probably consider the warning a bug and silence it properly. ^.^

grych · February 16, 2018, 5:11pm

Not in this case. Let me explain more detailed.

There are few cases, when Drab.Live can’t execute the AST (because of limitations). In this case, I catch CompileError exception, and re-raise it with better explanation, so user is not completely lost

And of course, I would like to test this case.

Anyway, looks like I need to live with the warning

OvermindDL1 · February 16, 2018, 5:17pm

Can’t you just do _u = u in the generated code?

grych · February 16, 2018, 5:26pm

Nope. u still does not exists

This is what I am testing:

<%= for u <- @users do %>
  <%= if u != @excluded do %><%= u %> <% end %>
<% end %>

When you update @excluded only, Drab tries to re-evaluate only that part: <%= if u != @excluded do %><%= u %> <% end %>. u does not exists there.

(BTW solution for this is there, it is to update @users and @excluded in the one poke, I just want to tell developer what to do - with better error message in CompileError).

ryanwinchester · October 15, 2018, 4:05am

Interesting, I’m @deprecating some stuff right now but still testing them, and the warnings in the tests are really annoying.

I wish we could do an assert_warn (similar to assert_raise) or something that “catches” it and stops it from being printed, which I could also use to check the deprecated message.

grych · October 15, 2018, 8:24am

Actually I resolved my case by completely rewriting this part of the library. This annoying warning encouraged me to do it, so I think warnings are useful. If you have one, try to find the other way to solve your issue.

ryanwinchester · October 16, 2018, 5:53pm

I want the warning, I just don’t want to see it in my console output in tests. It’s unecessary.

Capturing it and asserting that it’s there however, is what I’d really like to do. Similar to assert_raise.