How do programing languages talk each other through the Operating System?

I dont know how broad is this question. There are some elixir libraries to execute some heavy codes in another language. But how these work behind the scene? If i want to excute a code in another language wihout library, what do i need for this?

1 Like

In General:

There are two ways programs talk to each other:

  1. using streams (known as ‘pipes’) of textual or binary data.
  2. using a foreign function interface (FFI) where both languages pretend to be a common language, usually C.

Elixir-specific:

  1. There are ‘ports’, which is an external OS-program that is wrapped by a process. This is how Elixir wraps the ‘streams’ functionality.
  2. There are NIFs, (Native Implemented Functions) where Elixir’s VM callled the BEAM directly calls a C(or C-like, using an FFI) function, whose result might be transformed back into an Elixir datatype.

There is a 3. as well: It is possible for a program in another language to pretend to be a full-blown Erlang node with its own internal processes as well.

5 Likes

What is cons of calling another programing language ? Elixir developers write libraries instead of using a existed C library. Isnt easier using a strong C library for a spesific things? For example instead writing a json library in elixir Why not a strong C library ?

1 Like

Well, because a crash in a NIF can crash the whole BEAM, also in the early days (pre OTP 20), all native calls had to return in less than 1ms to not confuse the sceduler.

Also when you call into C, you always need to convert back and forth between erlang- and c-datatypes, which also has some overhead that is not alwys worth the effort.

Wrapping a C library and make it work with Erlang/Elixir, is tedious and sometimes needs more effort to put in, than simply to write similar functionality in erlang/elixir directly, as to write a NIF (or a binding to a native lib) you need to be fluent in at least 2 languages.

Last but not least, there is a big problem with cross platform compatibility when using NIFs. They often cause problems when one tries to build on windows. Or they blow up when the wrong library versions are installed on the host that runs the library later on.

8 Likes

Another thing is that the different ways of interfacing other languages have different “feels”. So using NIFs makes the interface very call-based where the Erlang/Elixir side calls the other language, while using ports makes the interface very message based where both sides send asynchronous messages to the other.

2 Likes

To cross programming language boundaries there needs to be a translation mechanism between both languages for the basic and constructed types. This is called marshaling and unmarshaling. E.g. In C/C++ you have short, int, long and the unsigned equivalent while in Elixir you have integer.

Then you need some means of informing the other side to execute some code, and its associated arguments.

Historic examples include: ASN.1, CORBA, COM, RMI, some still in use. Examples of modern attempts include: BERT, Thrift and Protobuf.

To get the request across the two common means are via sockets and pipes. Recently sockets seem more popular in comparison to pipes.

2 Likes

I like to also point out that if the target language runs on the JVM you can use JInterfaces in Erlang/Elixir to interact with that library. IMHO setting up JInterfaces is easier in comparison to NIF and ports.

1 Like