marcello

marcello

Why is data deep-copied when sending to another process?

Hi everyone,

I’m just learning elixir with the help of “Elixir in Action” by Sasa Juric. In Chapter 5 “Concurrency primitives” there is something I don’t get. Maybe someone can help me?

It’s with regard to deep-copy data when sending data to another process.

“You may wonder about the purpose of shared-nothing concurrency. First, it simplifies the code of each individual process. Because processes don’t share memory, you don’t need complicated synchronization mechanisms such as locks and mutexes. Another benefit is overall stability: one process can’t compromise the memory of another.”

Why should there be a need for sync. mechanisms? Data is immutable, it can’t be mutated by another process. So compromising data shouldn’t be possible.
What am I missing?

Thanks, Marcello

Most Liked

rvirding

rvirding

Creator of Erlang

I think one thing to realise wrt gc is that if you share data between processes then when you do a gc you have to gc all the processes and the whole heap. This means that you need a real-time collector which complicates things. Running in multiple threads which the BEAM does complicates matters even more as you need to make it thread safe and/or pay a large cost in synchronisation. This is now done for the large binaries which results that sometimes it can take long time to reclaim their memory. Basically every process which has referenced the binary has to do a full gc before the binary can be reclaimed. This can take a comparatively long time and overflowing memory with unreclaimed large binaries is actually a problem.

So while not sharing data and copying data when sending messages is actually quite a good way of doing it even if it sounds wrong. Way back when I did some implementations of erlang with shared memory and real-time collectors and it is not trivial to get it right. Fun though, but not trivial.

In 22 there are some new special memory areas which are shared so some things are good and really fast but you also pay a heavy price. Check out atomics, counters and persistent_terms and you will see what I mean.

rvirding

rvirding

Creator of Erlang

I think you will end up copying most of the data anyway but it a much less controlled fashion. As someone has pointed out you are viewing the data sent in a message as one big coherent chunk created in the sender and sent to the receiver, but this is generally not the case. Usually the data in a message is put together from smaller bits which come from many different processes. This will mean that either you have a right mess of keeping track of what and from where, or you will be doing a lot of copying between processes which is what you are trying to avoid.

A simple example of what I mean. Assume A sends a chunk of data to B which sends it on to C, so A → B → C. From what I understand of your method is that the data is not copied and A knows it has sent to B and B knows it has sent it to C. What happens if/when B dies? A does not know about C so if A now dies it can’t copy the data to C. Or should B tell A about C? But that means a process has to keep track of where all its data has come from, which we don’t need to do now.

This is exactly the problem you get with atomics, counters and persistent_terms. Atomics and counters solve the problem by only allowing integers while persistent_terms allow complex data structures but require a global gc when you update the data.

hubertlepicki

hubertlepicki

yes, precisely that. However, what @marcello says is true mostly for the small messages.

If you have a message consisting of binaries larger that 64 bytes (which is, relatively small), these will not be copied and instead will be allocated somewhere in designated space on the BEAM. So, if you are sending large blobs between processes these are not copied over.

Where Next?

Popular in Questions Top

nobody
Hi! In PHP: $_SERVER[‘SERVER_ADDR’] - in Elixir? Searched the docs for ip address and the web, no good results. Thanks!
New
Fl4m3Ph03n1x
About me? ( if you have nothing better to do than reading about some random guy in the internet :stuck_out_tongue: ) Hello all, this is ...
New
jay1
Why is it that the mnesia database isn’t the most preferred database for use in Elixir/Phoenix?
New
freewebwithme
Using vs code and installed ElixirLS: support and debugger. And I got an error popped up on start up says Failed to run ‘elixir’ comma...
New
SoCreat
i’m a new one to elixir which editor can i use vs code? or atom? Thanks! :smiley:
New
romenigld
I am trying to run a deploy with docker and I successfully runned with this command: docker build -t romenigld/blog-prod . but when I t...
New
svb
Hi! Currently I want to submit a form by pressing the Enter key. However, since my input field is of type “textarea” this is just adds a...
New

Other popular topics Top

nobody
Hi! In PHP: $_SERVER[‘SERVER_ADDR’] - in Elixir? Searched the docs for ip address and the web, no good results. Thanks!
New
minhajuddin
I have seen a lot of code which picks the first element from a list using Enum.at(0) instead of List.first. Is there a reason why people ...
New
Brian
What is the proper way to load a module from a file in to IEX? In the python world, doing something like this pretty standard: from ....
New
sergio_101
I am VERY much an elixir newbie. I have taken one elixir course and one phoenix course on Udemy. During that course, I saw the instructor...
New
shijith.k
I am trying to start a new phoenix project with elixir 1.9, but mix phx.new does not work. It says that ** (Mix) The task "phx.new" could...
New
dblack
I’ve got an issue with an app and I’ve no idea of how to troubleshoot it. I’m hoping someone here might have seen something similar. I p...
New

We're in Beta

About us Mission Statement