jfrolich
Distributed Elixir in Amazon ECS
Did anyone successfully set up ECS using distributed Elixir? I am tackling this right now.
I am especially interested how to connect the nodes. I was thinking it might make sense to write a custom libcluster strategy that fetches the current ip’s connected to the Application Load Balancer. ECS also offers service discovery using DNS, so that might also be a solution (this creates an A record for each node) and there seems to be a DNS strategy already in libcluster.
As far as I know these are the things to be addressed:
- Setting the node name to:
<APPNAME>@<PRIVATE IP> - Exposing the Port Mapper Daemon port in the VPC and docker image (4369)
- Exposing the intra-erlang communication ports (configurable using inet_dist_listen_min, inet_dist_listen_max) in the VPC and the docker image
- Service discovery (setting up communication between nodes) using a libcluster strategy
Setting the hostname to include the private IP can be done by using curl http://169.254.169.254/latest/meta-data/local-ipv4, not sure yet how to best inject this as a env variable.
Perhaps anyone already has figured some of this out on ECS/EC2? Help really appreciated! Planning to document the results so it’s easier for other to get this up and running. I already have some experience building a very small docker image using releases and multi-stage builds.
Most Liked
quolpr
Btw, now ECS support DNS service discovery out of the box.
I just used libcluster/lib/strategy/dns_poll.ex at master · bitwalker/libcluster · GitHub and it works well.
My vm.args:
-name <%= release_name %>@${PUBLIC_HOSTNAME}
-setcookie <%= release.profile.cookie %>
-kernel inet_dist_listen_min 9000
-kernel inet_dist_listen_max 9010
My start.sh:
#!/bin/sh
export REPLACE_OS_VARS=true
export PUBLIC_HOSTNAME=`curl http://169.254.170.2/v2/metadata | jq -r ".Containers[0].Networks[0].IPv4Addresses[0]"`
echo "Hostname: $PUBLIC_HOSTNAME"
REPLACE_OS_VARS=true /app/release/bin/start_server foreground
The DNS name can be taken from service details page. Usually it is my-service-name.local. Use it as query for Cluster.Strategy.DNSPoll strategy
drewble
I struggled with this for a week and eventually got it working. The information in this thread is enough to point in the right direction, but there is a big gap between the right direction and a successful deployment.
I’ve written up a fresh guide on deploying distributed Elixir to ECS Fargate tasks with Service Discovery and libcluster’s DNS Polling strategy. Hope it helps save someone some time!
schrockwell
You have hit on the major points. For the hostname, I can offer a concrete example. In the script that starts my application:
# Permit OS variable substitution for starting the VM
export REPLACE_OS_VARS=true
# Get the EC2 fully-qualified hostname for the node name
export PUBLIC_HOSTNAME=`curl -s http://169.254.169.254/latest/meta-data/public-hostname`
Then in rel/vm.args you can set the node name using the env var:
-name <%= release_name %>@${PUBLIC_HOSTNAME}
Popular in Questions
Other popular topics
Categories:
Sub Categories:
Forums
Popular Tags
- #ecto
- #liveview
- #troubleshooting
- #learning-elixir
- #deployment
- #library
- #erlang
- #testing
- #genserver
- #mix
- #absinthe
- #remote-other
- #otp
- #plug
- #how-to-question
- #macros
- #postgres
- #channels
- #elixirconf
- #exunit
- #discussion
- #code-sync
- #javascript
- #podcasts
- #onsite
- #dialyzer
- #docker
- #authentication
- #umbrella
- #full-time-contract
- #podcasts-by-brainlid
- #ecto-query
- #elixir-ls
- #phoenix_html
- #iex
- #blog-post
- #graphql
- #genstage
- #ai
- #websockets
- #supervisor
- #advent-of-code
- #elixirconf-us
- #distillery
- #processes
- #forms
- #api
- #metaprogramming
- #security
- #performance









