Hey folks,
Glad for being apart of Elixir community,
Elixir/Erlang is perfect choice for building Distributed systems, but can we do better? Yes by improving the network layer! Here is my draft concept :
our current internet model is end to end telephony system not suited for distributed environment, recently a lot of researchs for future internet infrastructure and the most interesting one is Named data networking (NDN) where the routing protocol shift from host centric(IP) to data centric(chunks) without the need to setup connections between two ends, NDN have two kind of packets (interest packet, data packet) anyone in the network can send an interest packet for a data and will receive back the data packet now the interesting part : NDN support native multicast, and also its mobility friendly (as there is no setup connection, the client can seamlessly enjoy a live stream and crossing border to other carrier without any cons)
this kind of network approach perfectly fit distributed systems, imagine an elixir/erlang cluster does not to setup connections where the whole cluster focus on the data it self.
What do you think guys? Do you see any advantage of using erlang/elixir with NDN approach and does the OTP fit well ?
Not sure if this going to be helpful to you or not:
In general, just get an idea and go build something small and useful with the tech that you think might provide a benefit over existing stuff, such as in this case with a different networking approach. If that’s possible in this case, at least, hadn’t heard about NDN so far.
From my perspective, unless there is significant motivation to use something, or get somewhere, or achieve any kind of (personal) benefit, there can be a lot of talk, but there will often be little value practical relevance.
Go build something! Especially if you’re excited about something. Then show us and a lot more people will be able to join in the conversation, and it’s going to have a practical direction!
Thank you for your kindness reply,
We live in a world where our communication paradigm is shiftting to data driven(many to many), so I posted this topic to see if already someone else try to do so.
Am still new to elixir, am trying to figure out if using NDN(ICN) will reduce the cost for elixir cluster, and example:
“Node A” request(send interest packet) a data from “Node B”. If “Node B” already sent the data before to “Node C” and “Node C” was closer to the edge router of “Node A” then its likely “Node A” will receive the data packet from “Node C” … because in-caching network feature, this likely will increase the High availability of the data to the requesters(server, clients).
My hope(general idea) is to free the servers from uncessery work if there is already requested data exist in caching network. Thank you
In (many) other languages, you might also be interested in IPFS or W3C Linked Data if you aren’t married to named data.
In Elixir, you just might be interested in ibGib. I’ve just stopped coding at the distributed networking layer, as I’m on my bicycle sojourn (currently 37°F in Mason, TX after c. 110 miles). The web app looks rinky dink as I’ve been coding solo on the entire stack, but it’s built on top of blockchain/merkle DAG/yada yada yada. The distributed aspect is unfinished, but it sounds like that is the area you would be interested in. The protocol is similar to the named data approach, but has stronger identification and integrity guarantees, as well as many other facets. Anyway, src is at ibgib.com/src .
Btw caching and cache invalidation were two primary requirements for the design for exactly the reasons you mention. But it can’t only be reactive, like “hey do you have data xyz”. It’s all about proactively getting AIs creating data projections before they’re requested…but I digress.
Either way, go for it. Named data is purdy cool stuff too.
This is really interesting, actually that’s why am learning Elixir to deeply understand blockchain and also convert my concept to a proof of concept, and the best way to do this is by developing it using amazing language like elixir( 2 in 1
I really like your project, the caching aspect is very important especially for decentralized environment,
Making the clients somehow act like a swarm and serve the data for each other it will be really cool. This also mean the server(cluster) will reduce the req/s ratio, why I am thinking in this? Think a real time video call from many to many, one server can’t handle it(it can if it have good transit link and powerful cpu), so instead of foucsing on increasing the req/s ratio per server, we just enable the trusted parties to have many to many video call on the edge routers(mobile edge computing application).
Am glad you mention DAG, recently am studying Bayesian network approach with micro detectors which will represent a graph that could be used to detect and prevent some kind of possible attacks for the cache within the network.
Thank you for the src am going to read it and go through it line by line.
Yes, exactly. This is what I call emergent load balancing. That’s actually a big selling point for the IPFS stack (love their work), but I see a better path forward with ibGib’s data protocol over ipld (even tho they did finally get named links). But I would point out to you that enabling “trusted parties” is a nontrivial problem. That’s where my design shines in it’s ability to unify hash-based nodal identity with ai/ml algorithms for dynamic network emergence.
Also, since you mentioned it, Bayesian inference/networks is only one implementation of an ML algorithm. And with ibGib, you could evolve entire ecosystems of varying algorithms, precisely as we currently evolve code bases manually in version control, e.g. git. This stems from ibGib’s semantic versioning.
Thanks for your kind words, best wishes, and let me know if you have any questions!
