Most tutorials and guides assume that you are writing a server where all of the users are going to create user accounts on the one server.
But what about each user running their own server from their computer and connecting to one another via “thin” relay servers?
How would the design of that system work with elixir? Each process would represent one “relationship” that you have to another user? So if you have 150 friends, you have 150 processes running on your machine, listening for updates?
Then the relay servers are for DHTs of usernames to addresses for connection?
Just wanting to start a design discussion on this since most material on Elixir assumes a very “web 2.0” model.
Would be doable. Your node would have to be online to send updates to your friends node when it comes online. Not sure of the implementation details of securing the networking and addressing but the relay server could probably just be where nodes register themselves / get a list of known nodes. Or really in the client you just identify as a master so you have a mesh of masters that keep a sync’d list of all known nodes where masters are known to be online most of the time.
I think actually it’s doable now. Certainly there are challenges, and some of this gets into experimental tech. Also, I am no expert in this area, so please correct me if I get anything wrong.
Raft and Paxos are CP systems (from CAP theorem), and what OP is asking about would need to be an AP system because, being peer-to-peer, nodes can come and go anytime. AFAIK, CP systems aren’t designed for this, but AP systems are. CP systems seem to be getting all the interest right now.
I’m working on a project called OpenPeerNetwork (OPN), which is an AP system built on Phoenix, and I’m writing a client for it in JavaScript and TypeScript. An important component to making this feasible, will be the handling of tracking causality or handling conflict resolution. For this, I’ll be evaluating the NDC framework, which is described in the research articles 1, 2.
I could use some help, so if you find this idea intriguing, please checkout our wiki for proposed architecture, and join us on discord.
Activity Pub (1) assumes a traditional server-client model where accounts live on the servers and users access those accounts through thin clients and (2) it has no concept of permissions, so “chat rooms” for example would need to be bolted on.
I’m bearish on ActivityPub in general for many reasons, but ignoring that, I just don’t think it’s worth the effort of maintaining compatibility with a system that is not intended for a use case like I’m imagining (real time communication).
I’m open to learning more about how ActivityPub can be abused to work as a general decentralized protocol, though!
If real-time P2P communication is what you are after, you could use WebRTC and have the large part of your logic in the browser. You would still need to implement signaling server-side (Phoenix channels would be great for that), but WebRTC comes with a lot of stuff already figured out (tunable reliability vs. performance, NAT traversal, encryption, audio/video, etc.) and runs on all modern browsers.
What would be extremely interesting is implementing WebRTC peers in Elixir, at least for data channels. Definitely not easy, but it would be amazing.
That’s what we’re doing in OPN. When two clients join the same topic, the server notifies each peer. The clients then send the offer and answer JSON through the server as a forwarded message – basically the same process you would use in a chat server for a private message.
This connection process all works now, but causes the client to crash at the moment, so needs more work. We need to implement the causality tracking that I referred to earlier, so the peers can sync efficiently. Once that’s done, I think that will also automate the coordination with the server. Meaning, if some data is sent from peer1, to both a peer2 and the server, peer2 doesn’t need to receive the update from the server, and I think that the NDC framework will automate this.
And, BTW, in OPN, a topic is generated for each subject-predicate pair for any given JSON blob. Meaning, global ID of the blob (this is the subject), truncated with the object property (this is the predicate). Which means the value of each property in the JSON object, is the object in an SPO triple. This is the scheme which allows us to have a topic for every single piece of data, and also how we map that data to either KV pairs (for ETS) or triples (for a triple store graph like Cayley).
Part of what I was wondering about is maybe we just layer a decentralized identity system on top of a global network of Janus servers that act as thin relays.
You should have a look at the openEngiadina CPub project then…
It implements the Client-to-Server (C2S) part of the ActivityPub specification, which allows the servers to be message relays in similar ways to XMPP. CPub also builds on the extensibility mechanism of AP supporting custom Linked Data vocabularies. An experimental feature is content addressing, which paves the road towards use in P2P scenario’s.
