A relatively shallow answer: this is basically a protocol that allows non-Erlang nodes to participate in Erlang distributed mesh. I heard of implementations in Go and JS.
This chapter of the docs talks about the different ways to interface with Erlang.
Erl_Interface
The program at the other side of a port is often a C program. To help the C programmer, the Erl_Interface library has been developed
The Erlang external term format is a representation of an Erlang term as a sequence of bytes, that is, a binary. Conversion between the two representations is done using the following BIFs:
Binary = term_to_binary(Term)
Term = binary_to_term(Binary)
A port can be set to use binaries instead of lists of bytes. It is then not necessary to invent any encoding/decoding scheme. Erl_Interface functions are used for unpacking the binary and convert it into a struct similar to an Erlang term. Such a struct can be manipulated in different ways, be converted to the Erlang external format, and sent to Erlang.
You can use file descriptors 3 & 4 instead of stdio over a port. See open_port docs for more info. If you use binary for a port Port.open({:spawn_executable, exe}, [:binary, packet: 4, ...], you encode with binary_to_term when you send to the port, and decode with term_to_binary when you receive. In the port, you have to encode/decode too.
erl_interface is a bunch of c functions for encoding/decoding external term format, communicating with nodes etc. Boilerplate if youâre writing something in C that you want to interface with Erlang.
You can implement the network protocol w/o using erl_interface, ala Pyrlang.
You donât need to use erl_interface to communicate to erlang/elixir through a port from the c side, if you use another serialization format that are understood by both, such as JSON.
With erl_interface, you can use the native erlang term format for port communication in the simplest case. In a much more complex case, you can even pretend to be a erlang node (c-node) like @dimitarvp mentioned.
erl_interface can be used for many different purposes. For example:
Write a program in C/C++ that behaves like a distributed Erlang node and communicates with other Erlang nodes.
Simplify the implementation of a port program and the Erlang code that communicates with the port program. The Erlang code can use just term_to_binary/1 when sending data and binary_to_term/1 when receiving data (as opposed to having the Erlang code use the binary syntax to decode and encode binary data). The same thing can be used in a driver.
Write a C/C++ program that reads a Wings 3D file, decodes the Erlang term inside it, and displays the 3D model or converts it to some other 3D file format. In the opposite direction, a C/C++ program could convert another 3D file format to a Wings 3D file.
Where I said Erlang, you can substitute Elixir or any other BEAM language.
@bjorng : Here is my XY problem. I have some code in Rust. I have some code in GoLang. I want to
use Elixir to provide a REPL and supervise/orchestrate/manage everything.
One obvious solution is:
use Protobuf + generate bindings for Go/Rust
use TCP for everything
for each GoLang/Rust kernel-process, have a corresponding Elixir âtranslatorâ process whoâs sole job in life is to translate Elixir data <-> Protobuf, and send it over TCP.
I am looking to see if there are better Elixir/Erlang based solutions. So far, I see: Nif, Port, erl_interface. erl_interface, as far as I can tell right now, looks like binary erlang terms over stdio.
Question: do you have any advice regarding this? I am increasingly leaning towards the protobuf/TCP solution, but I feel I donât understand the existing Erlang/Elixir solutions well enough to discard them.
I am not sure what you mean by âuse TCP for everythingâ. Do you mean Erlang distribution over TCP, or plain binary data over TCP with your own encoding/decoding code, or something else?
I donât know enough about Go and Rust, or how complicated data you will need to send, but here are some thoughts:
If it is possible to link the erl_interface library to Go and Rust executables (or if the Rust version of erl_interface supports Erlang distribution), you could have those executables pretend they are Erlang nodes. That will make the code on the Erlang/Elixir side very simple. It may make the Go/Rust code more complicated.
If you use a NIF (assuming that you can link Go code into a NIF; I know you do it with Rust), the Erlang system could crash or become unstable if there are any bugs in the Go/Rust code or any of your interface code). If that is problem, you should not use a NIF.
Whether you use port programs or a plain TCP socket doesnât really matter. Either way, you will need some way to encode/decode your data. You could use erl_interface on the Go/Rust side, and term_to_binary/binary_to_term on the Erlang side, or you could use any other way to encode/decode the binary data.
To some extent, which solution you choose depends on where you want to complicate the interface code. Using erl_interface will make the Erlang/Elixir code simple, but will probably make the Go/Rust code more complicated.
That will probably work well. I donât know much about protobufs, except that there is an excellent Erlang library for protobufs (which I assume you already know).
As a review of whatâs going on, I have some server components in GoLang. I have some server components in RustLang. I want to glue / manage everything from Elixir. The core question is: how does Elixir-Go communicate and how does Elixir-Rust communicate. For simplicity, letâs only consider the Elixir-Rust case.
known solutions
My known options are (a) NIF, (b) Port, (c) erl_interface, (d) protobuf over tcp.
question 2: does erl_interface have some epmd / run as independent node functionality ?
I have setup distributed Erlang with multiple nodes and multiple machines and played with epmd. erl_interface claims to make GoLang/RustLang code an âerlang nodeâ â but in the example above, all I see is binary-erlang-terms over stdio ports â i.e. the Go/Rust code is still âwrappedâ by an Erlang process.
Does erl_interface allow Go / Rust to run as âindependent Erlang nodeâ, without being first wrapped by an Erlang process ?
question 3: what is the advantage, if any, of erl_interface over protobuf-over-tcp ?
This is a really insightful point, and I am currently leaning heavily in favor of pushing all the complexity to Elixir, as Elixir is serving as the âglueâ.
I think most of that is answered above and in the links.
You can use ports without erl_interface, but youâll want a library in that language that will encode/decode the external term format. Go has at least one. Sasaâs post on ports is a nice primer.
This is dismissive and patronizing. What is the answer to q2? Can erl_interface allow Go/Rust programs to run as distributed Erlang nodes without first being wrapped by an Erlang process?
Both. Your choices are wide open. I will start from port and skip erl interface, just use json. This should be even easier than say protobuf. Once you hit some perf bottleneck you can always do more.
Using erl_interface, were you ever able to write a non-Erlang/Elixir program that functioned as an distributed Erlang node, without first being wrapped by an Erlang process ?
Your latency / throughput requirements != my latency / throughput requirements. As stated, most of the servers are already in GoLang / Rust.
Havenât try that. I guess it is pretty involved. I use ports. It is a library that contains the necessary functions; you link it with your program so there will be no Erlang process involved.
Of course. So you need to judge for yourself. The possibility is there.
Can you point me at demo code that does this: use erl_interface to have non-Erlang program act as a distributed Erlang node, without having it be wrapped by an Erlang process.