Libcluster in docker on aws elastic beanstalk - can't create a cluster

I can’t get my environment on aws elastic beanstalk to create a cluster. I am hoping somebody can help me here?

I have set up my project using a dockerfile, and deployed it to Elastic beanstalk. It’s deploying fine, but it’s not able to connect to other nodes. Receiving the following message:

20:47:23.103 [warn] [libcluster:example] unable to connect to :"my-api@172.xx.xx.xxx"
20:47:23.104 [warn] [libcluster:example] unable to connect to :"my-api@172.xx.xx.xxx"

There are two ec2 instances at the moment.

I added the following rules to the security groups inbound rules:

Custom TCP | TCP | 4369 | 0.0.0.0/0
Custom TCP | TCP | 9100 - 9155 | 0.0.0.0/0

Here is some other information I configured:

env.sh.eex

export RELEASE_DISTRIBUTION=name
export RELEASE_NODE=<%= @release.name %>@$(curl -s http://169.254.169.254/latest/meta-data/local-ipv4)

Dockerfile

FROM elixir:1.10.3-slim as builder
RUN mkdir /app
WORKDIR /app
ADD mix.* ./
RUN MIX_ENV=prod mix local.rebar
RUN MIX_ENV=prod mix local.hex --force
RUN MIX_ENV=prod mix deps.get --only prod
ADD . .
RUN MIX_ENV=prod mix release --overwrite

FROM elixir:1.10.3-slim
RUN mkdir /app
WORKDIR /app
RUN apt-get update && apt-get install -y curl
COPY --from=builder /app/_build/prod/rel/my-api .
EXPOSE 4000
CMD bin/my-api start

Dockerrun.aws.json

{
  "AWSEBDockerrunVersion": "1",
  "Image": {
    "Name": "xxxxxx",
    "Update": "true"
  },
  "Ports": [
    {
      "ContainerPort": "4000"
    }
  ]
}

config/prod.exs

config :libcluster,
       topologies: [
         example: [
           strategy: ClusterEC2.Strategy.Tags,
           config: [
             app_prefix: "my-api",
             ec2_tagname: "elasticbeanstalk:environment-name"
           ],
         ]
       ]

I also hardcoded the RELEASE_COOKIE.

I’ve never done it with Elastic Beanstalk, but I have gotten AWS Service Discovery to work with ECS Fargate and libcluster using this:

topologies = [
      ecs_app: [
        strategy: Cluster.Strategy.DNSPoll,
        config: [
          polling_interval: 1000,
          query: "phx-fargate.phx-fargate.local",
          node_basename: "phx_fargate"
        ]
      ]
    ]

Not sure if the DNS strategy might work better for you there. See this thread for more details: Distributed Elixir in Amazon ECS for more discussion. With AWS offering Service Discovery out of the box, you no longer need to mess around with A records and DNS entries. This is my config in the Service in ECS.

Screen Shot 2021-02-08 at 5.48.54 PM

Hope this was helpful.

5 Likes

This was very helpful. Moved it to ECS using service discovery.