Notify origin of messages on :peer nodes in distributed tests

Questions / Help
, ,
1

It seems the latest and greatest way to write distributed tests in elixir and erlang is to use the :peer module with the connection: :standard_io since this allows you to spin up peers and communicate with them via standard_io rather than making your test node part of a distributed cluster. This allows you to test things like network partitions, and keep things async since each test can have its own protected cluster.

That being said, the api to communicate with those peers seems fairly limited to send, call and cast. In my tests, I need to notify the test process when messages are received on one of the peers. For instance with a normal distributed cluster you could simply send a message back to your test process like

    # Note: untested pseudocode as an example
    parent = self()

    Node.spawn_link(node, fn ->
      Worker.do_stuff()

      next_message = fn ->
        receive do
          message ->
            send(parent, message)
            next_message.()
        end
      end

      next_message.()
    end)

    assert_receive :initialized

    :rpc.call(node, Worker, :do_other_stuff, [])

   assert_receive :new_message

With the peer connection, you can’t send messages directly to the host/origin node since it’s not part of the erlang cluster.

When you’re just waiting for a single message, in the past I’ve used :peer.call with a quoted block of code to execute like this…

      code = 
        quote do
          Worker.do_stuff()

          receive do
            :initialized -> :initialized
          end
        end
      
      :initialized = :peer.call(node_pid, Code, :eval_quoted, [code]

However, this only works if you’re waiting on one message and then return it. In my situation I want to continuously monitor for incoming messages and forward them to the test process so that I can make assertions about messages that come in. Any ideas on how to make that happen with :peer on standard_io connection?

1 Like