fireproofsocks
Simple examples of distributed systems in Elixir?
I’m working to increase my understanding of how to build distributed systems as a way to leverage concurrency. I understand this is a broad topic, but I’m wondering if anyone has some examples of how to structure “work” across multiple nodes or multiple apps.
I admit I am not even sure I follow how things happen when you deploy a single app across multiple nodes (e.g. 1 app deployed to 2 EC2 instances)… that means there are 2 instances of the Erlang VM running. How is work delegated? E.g. if it’s a Phoenix app, does a multiple-server deployment assume that a load balancer is figuring out the “division of labor” and routing requests to different nodes? And does this assume that each node more or less has no awareness of the other nodes?
Or how about when you want to process a stream of inputs (e.g. using Task.async_stream/3)… do you have any way of routing the processing to different nodes? For this to work, wouldn’t each node need to know about the other nodes? Locally, this happens by specifying --sname and --remsh options when starting the app, e.g. something like
iex --sname app1@localhost -S mix
iex --sname app2 --remsh app1@localhost
What are good questions to ask when deciding how to break up work across multiple nodes/apps?
Most Liked
viniciusmuller
Hey, have you checked the “Introduction to Mix” guide in the official documentation?
Although the name can be a bit misleading, this is the description of the guide:
In this guide, we will build a complete Elixir application, with its own supervision tree, configuration, tests, and more.
The application works as a distributed key-value store. We are going to organize key-value pairs into buckets and distribute those buckets across multiple nodes. We will also build a simple client that allows us to connect to any of those nodes and send requests
I think it might be helpful, it covers how nodes connect to each other using the configurations and shows some example of running code in a remote node and distributing work across nodes via sharding.
Overall the erlang runtime does no magic to help you and you still need to build your own mechanisms of distribution and load balancing in your application, but it provides all the building blocks you’ll need to do so.
apsori
Popular in Discussions
Other popular topics
Chat & Discussions>Discussions
Latest on Elixir Forum
Sponsor Spotlight
Develop your skills with books, videos, and courses.
Categories:
Sub Categories:
Forums
Popular Tags
- #ecto
- #liveview
- #troubleshooting
- #learning-elixir
- #deployment
- #library
- #erlang
- #testing
- #genserver
- #mix
- #absinthe
- #remote-other
- #otp
- #plug
- #how-to-question
- #macros
- #postgres
- #channels
- #elixirconf
- #exunit
- #discussion
- #code-sync
- #javascript
- #podcasts
- #onsite
- #dialyzer
- #docker
- #authentication
- #umbrella
- #full-time-contract
- #podcasts-by-brainlid
- #ecto-query
- #elixir-ls
- #phoenix_html
- #iex
- #blog-post
- #graphql
- #genstage
- #ai
- #websockets
- #supervisor
- #elixirconf-us
- #advent-of-code
- #distillery
- #processes
- #forms
- #api
- #metaprogramming
- #hex
- #security
Our Sponsors
Build Elixir applications with speed and confidence.
Supporting innovation across the BEAM ecosystem.
We build reliable cloud platforms for business-critical systems.
Catch errors, track performance, monitor hosts and more.
Error tracking for Elixir devs who love to ship. Start your free account.
Practical resources that improve the lives of professional developers.
Develop your skills with books, videos, and courses.
Producing high quality Elixir screencasts since 2017.
The team behind Membrane, Popcorn, LiveDebugger. Available for hire.
Courses that'll move you from confusion to "Aha, now I get it!"