OTP in Unequal Trust environments

Defining unequal trust.

Consider three machines:

my_desktop = home desktop machine
ec2_node = node I am renting on EC2
bob_colo = node I am renting from @ Bob’s colo

My trust for these machines are my_desktop > ec2_node > bob_colo. In particular, my_desktop can ssh into ec2_node / bob_colo, but neither can ssh into my_desktop.

OTP, Distributed Erlang/Elixir

I believe that to take advantage of OTP in a multi-machine environment, we need to use Distributed Erlang/Elixir.

However, they have a different trust model from me – any machine can spawn any code on any node.

I don’t have that trust model: ec2_node and bob_colo should take commands from my_desktop; but my_desktop definitely should not take commands from ec2_node and bob_colo.


In an “unequal trust” environment, is it still possible to use OTP ?


Use OTP - yes, use Distributed Erlang - not really. The question is whether you do not trust machine or network? Because with network we can handle, with machine not really. You will probably need to use different distribution protocol that will provide “firewalling” some actions.

1 Like

I don’t trust the network, that is easy to solve (tunnel over ssh, vpc, etc …).

I do not trust ec2_node / bob_colo to issue spawn commands to my_desktop – as stated above, twice.

In theory, you are right. In practice, do you know of any pre-existing library that can be used ?

I am not aware of any. You may need to write one.

Depends on what you mean by “use”. Any given node can of course use OTP constructs internally. If you want to do standard erlang distribution between these nodes however then no, that will not be safe. I’d fall back to more standard collaboration / communication protocols.


Maybe you can do some type of client / server topology with Partisan GitHub - lasp-lang/partisan: High-performance, high-scalability distributed computing with Erlang and Elixir. ?

@benwilson512 @mpope : Unfortunately, it does appear the best that can be easily done is:

  1. distributed erlang for all nodes in the same trust level
  2. tcp for erlang nodes in different trust levels

Fundamentally, erlang cookies are a symmetric key, whereas we want an asymmetric trust model.

Erlang cookies aren’t security measure at all, and you can have different cookies for different nodes.

The problem isn’t cookie, the problem is that as soon as you connect to the cluster, then all nodes in cluster can execute any code on any node within that cluster. And it is not only about executing Erlang code, but any code (as you can always open port on remote node).


This is known. See original post:


The point here is: the trust model we want to express is asymmetric. Erlang cookies are symmetric. Therefore Erlang cookies can not express the trust model we want to express. As stated:

In “Programming Erlang” by Joe Armstrong, he writes:

Distributed Erlang applications run in a trusted environment—since any
node can perform any operation on any other Erlang node, a high degree
of trust is involved. Typically distributed Erlang applications will be run
on clusters on the same LAN and behind a firewall, though they can run
in an open network.


You could probably use Whonix/Qubes to run the “home” Erlang node in an isolated VM. Then you wouldn’t have to trust it.