Assert condition with timeout

It is definitely an anti-pattern as you will always 100ms even if the condition you are asserting on would be true after 1ms.

Ideally you want to assert_receive and assert on messages or perform a blocking call. For example, if you are talking to a server that performs async actions, you could have a sync call that simply returns ok. That in itself will guarantee all async messages have been processed.

Agreed, the sleep workaround is an anti pattern. I was thinking more in the lines of the following

defmacro await_assert(assertion, timeout) do
  tmp_module_name = "AwaitedAssertion" <> String.replace(to_string(:rand.uniform()), ".", "")
  quote do
    defmodule :"#{unquote(tmp_module_name)}" do
      def test_assertion(t) when t <= 0 do
        assert false, "Operation took too long."
      end

      def test_assertion(t) do
        try do
          unquote(assertion)
          :ok
        rescue
          _ ->
            :timer.sleep(1)
            test_assertion(t - 1)
        end
      end
    end

    apply(:"#{unquote(tmp_module_name)}", :test_assertion, [unquote(timeout)])
  end
end

In that way I would evaluate the condition every millisecond until a timeout is reached after which I fail the assertion. Would you still call this an anti-pattern? An example for a use case would be an asynchronous action that triggers the insertion of a record being inserted into an ets table which I would like to verify.

assert_receive/3 has already been pointed out to you - have a look at it and it’s friends.

• “Busy wait” is the anti-pattern - it doesn’t matter if the wait is 100 or 1 ms.
• In an efficient system events worth observing need to be broadcast - not “watched”. Interested parties should be able to register (and unregister) their “interest” so that they can be notified when the event occurs. That notification (event message) can then be used with assert_receive/3.

The fact that you need to structure a test in a “busy wait” manner may suggest that you have a design issue.

What is that ETS table used for? Typically they exist to support some sort of request served by the process. If the same process sends the async insert followed by a synchronous request then the request should be served after the insert has been processed - so the result of the request should be consistent with the insert being successful.

I still have to disagree here. assert_receive/3 is great for many use cases but not suitable for everything. Take as example a request that triggers the insertion of data into a cache. I want to assert now that after the request, data eventually is served from the cache. In this case I would not want to add a broadcast mechanism to the cache that informs interested parties about insertions and invalidations a) because it is expensive and unnecessary (except for testing) and b) because that would be a very different test.

This is no different than logging. Write your ETS access functions in such a way that for testing they compile to send a digest message to a named process for testing. When compiling for production that code can then be safely omitted.

elixir/lib/logger/lib/logger.ex at v1.5.3 · elixir-lang/elixir · GitHub

That is a nice idea. Thanks. Macros still amaze me. Here is a little code I’ve been playing around with to implement the suggestion:

defmodule TestMessage.Registry do
  @moduledoc false

  @registry TestMessage.Registry
  @default_key :test_message

  def name, do: @registry
  def default_key, do: @default_key

  def init do
    {:ok, _} = Registry.start_link(keys: :duplicate, name: @registry, partitions: System.schedulers_online)
  end

  def subscribe(key \\ @default_key) do
    {:ok, _} = Registry.register(@registry, key, [])
  end

end

Initialize the test message registry in the test_helper.exs as

TestMessage.Registry.init()

And register for test messages via the subscribe function within test cases. To send test messages use the following module that implements @peerreynders sugestion of checking that the code is compiled for testing

defmodule TestMessage.Sender do
  @moduledoc false

  alias TestMessage.Sender
  alias TestMessage.Registry, as: TestMessageRegistry

  defmacro send_test_message(message) do
    quote do
      Sender.send_test_message(TestMessageRegistry.default_key(), unquote(message))
    end
  end

  defmacro send_test_message(key, message) do
    if Mix.env == :test do
      quote bind_quoted: [key: key, message: message] do
        Registry.dispatch(TestMessageRegistry.name(), key, fn entries ->
          for {pid, _} <- entries, do: send(pid, message)
        end)
      end
    end
  end

end

Thanks again. This has been really helpful.

Just another idea, following @josevalim rationally and with no external dependencies. Code isn’t readable since Erlang trace API is complicated. But if you abstract the complication away in test helper functions, it can become manageable…

pattern = {Module, :function, 0} # it can be any {mod, fun, arity}
:erlang.trace_pattern(pattern, [{:_, [], [{:return_trace}]}])
:erlang.trace(:all, true, [:call])

# do async work that ends up calling Module.function()

assert_receive {:trace, _pid, :return_from, ^pattern, {:noreply, _state}} # block until Module.function is called

# write the rest of your assertions here, the ones that need `Module.function` to be returned...