agiuliano
Process distribution, communication and deployment
Hi guys, I’m exploring Elixir for high demanding systems and I’m trying to figure out some scenario I face every day with the software I manage.
When I deploy an Elixir app, I can connect nodes together to create a cluster: is there any limitation on the number of nodes that can connect to each other?
My understanding is that nodes, to check other nodes are alive, send small messages, a kind of heartbeat protocol: does this have impact on big clusters? I’m scared that this communication will also start to be visible in terms of resources the bigger my fleet is. What are the consequences in such an environment?
When processes are connected, I can use OTP to send messages among them, messages will be delivered on the remote node possibly. If I’m deploying the receiver node, I will get a failure in the sender node. Is this a situation I have to managed by myself, or are there any tools I can leverage?
I was reading an interesting post regarding message passing across nodes: sending messages real time in system with millions of requests per second is realistic with Elixir/OTP?
Thank you
Most Liked
tty
I’m not sure why but it has been often requested over the years to have a command/data channel separation. This is especially true once you throw Mnesia into the mix and pass around large datasets.
idi527
Yes, according to some anecdotal evidence from the heavy users, it’s about 60-200 nodes for the default “distributed erlang” topology (mesh). To go beyond that, check out either partisan [1] [2] or scalable distributed erlang (don’t know if it’s usable yet).
[1] GitHub - lasp-lang/partisan: High-performance, high-scalability distributed computing for the BEAM. · GitHub
[2] [1802.02652] Partisan: Enabling Cloud-Scale Erlang Applications
chrismcg
BEAM clusters are fully connected so when you connect a node into a cluster it connects to every other node and gossips with them. From the paper you linked to:
First, maintaining a fully connected mesh of Erlang nodes means that a system with n nodes must maintain O(n^2) active TCP/IP connections and this induces significant network traffic above 40 nodes
Popular in Questions
Other popular topics
Latest on Elixir Forum
Sponsor Spotlight
Real-time error tracking, performance insights, and observability for devs.
Categories:
Sub Categories:
Forums
Popular Tags
- #ecto
- #liveview
- #troubleshooting
- #learning-elixir
- #deployment
- #library
- #erlang
- #testing
- #genserver
- #mix
- #absinthe
- #remote-other
- #otp
- #plug
- #how-to-question
- #macros
- #postgres
- #channels
- #elixirconf
- #exunit
- #discussion
- #code-sync
- #javascript
- #podcasts
- #onsite
- #dialyzer
- #docker
- #authentication
- #umbrella
- #full-time-contract
- #podcasts-by-brainlid
- #ecto-query
- #elixir-ls
- #phoenix_html
- #iex
- #blog-post
- #graphql
- #genstage
- #ai
- #websockets
- #supervisor
- #advent-of-code
- #elixirconf-us
- #distillery
- #processes
- #forms
- #api
- #metaprogramming
- #security
- #performance
Our Sponsors
Build Elixir applications with speed and confidence.
Supporting innovation across the BEAM ecosystem.
We build reliable cloud platforms for business-critical systems.
Catch errors, track performance, monitor hosts and more.
Real-time error tracking, performance insights, and observability for devs.
Practical resources that improve the lives of professional developers.
Develop your skills with books, videos, and courses.
Enabling companies to succeed by building software people love.
Courses that'll move you from confusion to "Aha, now I get it!"