Process.send_after: send order for messages with timeouts expiring at the same time

We know that message receive order is guaranteed to match the send order. But how send order is defined when using Process.send_after for following situations?:

  • process schedules 2 messages at different points in time, but their timeouts expire at the same time
  • process schedules 2 messages at the same time with the same timeout (two Process.send_after calls in a row)

E.g.:

  • schedule sending message A after 1000 ms
  • wait 500 ms, then schedule sending message B after 500 ms and message C after 500 ms

Timers will be expiring “at the same time”, but how the actual send order is defined for messages A, B and C?

Does it always match the call order of Process.send_after?

1 Like

You are in a very good position to try it and let us know. :smiley:

I’ll try but this is more about order as defined by language. But I think this is the call order of Process.send_after, because anything else wouldn’t make much sense

I would say it can be any order because super precise timing has never been the focus of most operating systems except real-time hardened distributions of Linux, *BSD, Illumos and the like. So I’d think if you schedule something for the exact same timestamp then the messages can arrive at any order.

Which raises the question why would you rely on a particular order of messages? There are other ways to break down work without relying on fragile mechanisms like exact timings.

1 Like

I agree that depending on the order is bad and will look into how I can make order irrelevant in my use-case.

Also found this here:

assuming that the system is not heavily loaded, a timer will typically be triggered in the range [T , T+1 ) milliseconds when the user has given the timeout time T . If the system is heavily loaded, it may take an even longer time until a timer is triggered.

Pure speculation:

My main line of thought was that internally there should be some mechanism to handle such “same time” situations, or more generally to handle scheduled timers. E.g. in JavaScript there’s task queue which determines the order of scheduled operations in the order of insertion into the queue, so for identical timers the order is defined by position in the queue. I thought that there’re not much other ways to handle things like this regardless of the language and speculated that Erlang does something similar. So in case of heavy load situation the first timer would be delayed say for 50 ms, but so would be the next timer after it, thus the order will be preserved, despite that the actual trigger time could considerably differ from the user-specified value.

In short: I can see how the timer precision can be affected, but I can’t see how/why the ordering can be affected

For timers that are strictly and provably scheduled for different times I’d say your intuition is correct and that they’ll indeed come one after another, even if they are technically late (i.e. shifted from their original scheduled time).

But again, if you schedule 5 timers to fire exactly at 2030-01-01_13:40:17.100345 then I’d think you get zero guarantees which of the 5 will fire first.

And this is not about the Erlang VM per se, by the way. Long long time ago, almost feels like a previous life, I worked on drivers and the exact timings of things were impossible to catch, like trying to bottle a shadow. And I heard repeatedly that this situation has not improved to the point of actual full determinism, to this day.

IMO let us know what problem are you solving and we can suggest a way that does not involve relying on something that the lowest of programming languages and OS-es cannot solve for yet.

1 Like

Ordering depends complete fully on the implementation details of the sender.
How does the implementation trigger the timers. In what order does it do this? Is the sender 1 process or multiple?

In the Erlang Reference Manual there is this description:

The clock service, the name service, the timer service, and the spawn service mentioned previously are services provided by the runtime system. Each of these services consists of multiple independently executing entities. Such a service can be viewed as a group of processes, and could actually be implemented like that. Since each service consists of multiple independently executing entities, the order between multiple signals sent from one service to one process is not preserved. Note that this does not violate the signal ordering guarantee of the language.

Scroll down a bit after Sending Signals.

5 Likes