Trying to run an elixir cluster using AWS Elastic Beanstalk and libscluster

Currently at my company, we use AWS EB with Docker in order to deploy and autoscale our main application.

We have our instances inside the same VPC and allowed all traffic through their security groups, we are trying to connect them using libcluster’s Gossip strategy but still no success (the different instances can ping each other, though).

It seems that there is a limitation where we cannot expose more than a single port easily from our application’s container, thus explaining why the nodes are not connecting (we cannot expose epmd, for example).

Right now we are considering switching to ECS.

If anyone has made the EB → ECS transition or have a working production elixir cluster using AWS EB, we would really appreciate some feedback about the setup process that was required.

I would definitely recommend ECS (or Kubernetes for more demanding apps).

Here is a complete example of deploying an Elixir app to ECS using Terraform: GitHub - cogini/multi-env-deploy: Complete example of deploying complex web apps to AWS using Terraform, Ansible, and Packer

And here is an optimized container build and test process for a Phoenix app using Github Actions which deploys to AWS ECS: GitHub - cogini/phoenix_container_example: Full featured example of building a container for an Elixir Phoenix project, taking advantage of BuildKit caching and multi-platform builds (Arm). Shows raw docker, docker-compose, and Earthly; mirrored base images from Docker Hub to AWS ECR; deploys to AWS ECS using CodeBuild / CodeDeploy

Here is a similar, somewhat cleaner, example for Absinthe:

4 Likes