Are messages sent to self() immediately received?

According to this response from @NobbZ , this elixir function:

fn ->
    send(self(), :hello)

    receive do
      msg -> {:received, msg}
    after
      0 -> :not_received
    end
end

Will always return {:received, msg}.

However I couldn’t find any documentation to prove it. Is the BEAM guaranteed to work this way ? Or is it the result of an implementation detail which I should avoid to rely on because it might be subject to change ?

Put another way, is this a reliable test:

test "callback does NOT run" do
    fun_that_should_not_run_callback(fn ->
        send(self(), :cb_run)
    end)

    refute_received(:cb_run)
end

Thank you.

2 Likes

A single process works sequential on the beam, concurrency only exists between processes. Given the code sends a message to itself and then next checks the message queue for contents you can be sure that there’s at least that message it sent itself in the message queue and the message queue won’t be empty. There’s noone else taking messages out of the message queue. There may be other parties adding additional messages to the processes message queue though.

5 Likes

In the doc of receive/1:

In case there is no such message, the current process hangs until a message arrives or waits until a given timeout value.

And, in a process, it is a land of sequential programming, so the code will run sequentially.

After knowing all of these, let’s answer the question.

Yes. send(self(), :hello) sends a message to the mailbox of current process. receive/1 get the message out of the mailbox of current process. So, it will always work.

1 Like

There’s one thing to watch out for: a receive with a simple msg -> clause will pull the next message out of the mailbox, whatever it is. If you used that function in a situation where self() was being sent lots of messages (a busy GenServer, for instance) the unrestricted receive might get something unexpected:

iex(1)> send(self(), :wat)
:wat

iex(2)> fun = fn ->
...(2)>     send(self(), :hello)
...(2)> 
...(2)>     receive do
...(2)>       msg -> {:received, msg}
...(2)>     after
...(2)>       0 -> :not_received
...(2)>     end
...(2)> end
#Function<43.3316493/0 in :erl_eval.expr/6>

iex(3)> fun.()
{:received, :wat}

iex(4)> fun.()
{:received, :hello}

iex(5)> fun.()
{:received, :hello}

iex(6)> flush()
:hello

Here the extra :wat message was already in the mailbox before the first execution of fun, so it’s the first thing returned.

The idiomatic way to avoid these kinds of headaches is to match in receive, which will find the first matching message in the mailbox. For instance, refute_received generates code like this for the assertion in your test:

receive do
  :cb_run = actual -> flunk("Unexpectedly received message #{inspect(actual)} (which matched :cb_run")
after
  timeout -> false
end
9 Likes

Yes, this is something you really have to be aware of:

  • Sending a message to a process ALWAYS, I mean ALWAYS, puts the message at the end of the process’ message queue There is no way around this irrespective of to which process you send the message. There is no priority send.

  • receive with an unbound variable as a pattern will ALWAYS receive the first message in the queue.

This means that if you want/need to do priority send/receive you explicitly have to do it “by hand”.

6 Likes

Yes, but this won’t always be {:received, :hello}


Are messages sent to self() immediately received?

They’re received, but not immediately. There may be a context switch between send and receive.


test "callback does NOT run" do
    fun_that_should_not_run_callback(fn ->
        send(self(), :cb_run)
    end)

    refute_received(:cb_run)
end

No, this is not reliable test. To make it reliable, add a reference to the sent message

4 Likes

But what makes me sure that the message won’t be delayed ?

Who does the job of putting the message in the message queue ? Is it the sending process itself ? I always (naively) thought that another process of the BEAM took care of exchanging all the messages…

What makes the test unreliable ?

Do you mean the message could be sent but refute_receive will miss it ?

Our only guarantee is that signals (messages, links, monitors, et cetera) arrive in the order they were sent between process pairs. That is, if we have processes A, B, and C:

A sends message {hello,1} to C
A sends message {hello,2} to C
A sends message {hello,3} to B
B, as a response to the {hello,3} message, sends it along to C

Then {hello,1} will always arrive at C before {hello,2} does, but {hello,3} could arrive at any point, and even “time travel” before {hello,1}.

The guarantee doesn’t distinguish between sending to yourself or another process, and the sequence above can happen even if process C is A.

Edit: As per the documentation:

The only signal ordering guarantee given is the following: if an entity sends multiple signals to the same destination entity, the order is preserved; that is, if A sends a signal S1 to B, and later sends signal S2 to B, S1 is guaranteed not to arrive after S2. Note that S1 may, or may not have been lost.

The receive in the example accepts any message that is already in the queue, which is not necessarily the message you sent on the line just before.

11 Likes

I mean that message could be sent by another process

But is there any guarantee that there is a message in the queue or could the message queue still be empty then the receive is executed? E.g. imagine the code example is run in a process, which doesn’t receive any additional messages from other processes, is there a guarantee that the receive will pick up the message sent before, or could the queue be empty at that point?

1 Like

Yes, there is such a guarantee. This is only false for something like Process.send_after

1 Like

The only guarantee we give is that if you send two (or more) signals from one specific entity to another, the second signal will not arrive before the first. The first signal may never arrive, but if it does, it will never come after the second.

In the absence of anything drastic happening (e.g the process getting killed), the receive will match the message.

Whether the message was in the queue or not prior to the receive doesn’t make a difference since the only observable effect is that we (eventually) get a matching message. As an application developer there’s no reason to care (other than curiosity of course :slight_smile:), which gives us more leeway to optimize things than if we had documented the current implementation details.

4 Likes

I don’t see any case where

send(self(), :something)
receive do
  :something -> :ok
  after 0 -> :error
end

Returns :error. The message may not arrive here only if the process dies during send (like message queue memory overflow). But the message will always be in the message queue after send, even if this send yields due to reductions.

Perhaps you can give an example of the situation where the code above returns :error (without considering non-default OTP or ERTS implementations)

1 Like

We don’t guarantee that it will work the way you think. The current implementation may never lead to returning ‘error’, but we’ve left the option to do so open for the future.

1 Like

Thank you, by the way, I just thought of a new optimization that uses this fact to make demonitor-with-flush faster. :smile:

7 Likes

Thank you. I feel like my original question wasn’t very clear, but that’s exactly what I wanted to ask.

1 Like

Thank you.
That’s the impression I had but I wasn’t sure.
I found many pieces of code that use it, but no documentation that it would always be reliable.
So that’s why.

2 Likes

To sum up:

I wanted to know if this is reliable:

test "callback does NOT run" do
    fun_that_should_not_run_callback(fn ->
        send(self(), :cb_run)
    end)

    refute_received(:cb_run)
end

As noted by @hst337 , it’s not, because another process could have sent :cb_run by the time refute_received() run.

And also, depending of what fun_that_should_not_run_callback does, the callback function could be transferred to another process. send(self(), {:cb_run, ref}) would then run in that other process and be received by that other process, and so this test proves nothing.

This would be a better test:

test "callback does NOT run" do
    ref = make_ref()
    test_process = self()
    fun_that_should_not_run_callback(fn ->
        send(test_process, {:cb_run, ref})
    end)

    refute_received({:cb_run, ^ref})
end

But it was not the point of my question.
In fact, I wanted to know if a possible message sent by send(self(), {:cb_run, ref}) would necessarily be received by refute_received(), or if it could be delayed on the way and arrive too late.

Put another way:

fn ->
    send(self(), :hello)

    receive do
      :hello -> :ok
    after
      0 -> :error
    end
end

Is this function guaranteed to never return :error ?

Turns out that it works this way in the current BEAM implementation, but it is an implementation detail and there is no such guarantee.

An option is to use refute_receive/3 instead of refute_received/2:

test "callback does NOT run" do
    ref = make_ref()
    test_process = self()
    fun_that_should_not_run_callback(fn ->
        send(test_process, {:cb_run, ref})
    end)

    refute_receive({:cb_run, ^ref}, 100)
end

This is reliable because refute_receive will wait for {:cb_run, ^ref} for 100 ms, the disadvantage being that the test will last at least 100 ms.

Another option is to rely on the only guarantee we have, which is that the message order is guaranteed for a given pair of processes:

test "callback does NOT run" do
    ref = make_ref()
    test_process = self()
    fun_that_should_not_run_callback(fn ->
        assert test_process == self()
        send(test_process, {:cb_run, ref})
    end)
    
    ref_2 = make_ref()
    send(self(), ref_2)
    assert_receive(^ref_2, 100)

    refute_received({:cb_run, ^ref})
end

It will be faster, because assert_receive will not systematically wait 100 ms, but only until the ref_2 message arrives. Then, the {:cb_run, ref} message has necessarily arrived if it has been sent.

The disadvantage of this option is that you have to make sure that the function is not executed from another process, hence the assert test_process == self() line. Depending of what fun_that_should_not_run_callback does, it is applicable or not.

2 Likes

You have to note that while send is asynchronous, receive isn’t. It blocks the process until a clause matches or timeout.

I think what you’re missing here is the part about receive blocking the process. send may or may not actually deliver message before your process reaches receive, but since receive is blocking you can make it wait until the message is received (or timeout occurs, whatever suits you). This is also the reason why a general msg match may not actually work in your code snippet. Making your messages more specific makes receive block until it’s actually delivered.

EDIT: I didn’t mean send is async, I meant delivery is async. Words matter I guess.

1 Like