Reading “Implementing Riak in Erlang: Benefits and challenges”
Something got my attention,
I haven’t had the opportunity to fully comprehend this by actual work experience since most applications are not at Whatsapp or Riak scale.
Is that still true today, I know that the Erlang team has done huge improvement since 2014, but I am not sure where if this slice holds true today, or to what extent it does?
I think it refers to the fact that “cross node” message passing is basically always
copies the data. While inside the node we know that for example big binaries are stored in the shared heap and with the message to the process located in the same node only reference is passed, we cannot do that across nodes.
About multi-core erlang I have this paper in my bookmarks, but haven’t get to it yet personally: Characterizing the Scalability of Erlang VM on Many-core Processors
The problems with sending large terms over distribution were solved with this change in 2019:
02:35PM - 05 Feb 19 UTC
This PR implements fragmentation of large Erlang Distribution signals in order t
In practice people send tons of data over distribution and it works fine nowadays. RabbitMQ for instance is a heavy user. Phoenix Pub/Sub as well.