What is the best way of running a Phoenix server in cluster mode in AWS ECS?

Phoenix Forum Questions / Help
, ,
1

What is the best way of running a Phoenix server in cluster mode in AWS ECS?

It seems to me like a common problem. I plan to use a libcluster library, but I’m wondering if Phoenix provides anything out of the box.

Also I’m not a devOps, so I don’t completely understand why libcluster offers UDP multicast to 233.252.1.32 for a Gossip strategy… All I found so far are blog posts about creating a lambda that registers new nodes in a cluster, or using some patterns in naming EC2 instances, etc. Any recommendations?

4 Likes
2

Here’s my config block:

config :clove, :cluster_topology,
  main: [
    strategy: ClusterEC2.Strategy.Tags,
    config: [
      # EnvMap is just System.get_env or raise
      ec2_tagname: EnvMap.fetch!("LIBCLUSTER_EC2_TAGNAME"),
      # Should match what's in vm.args / distillery
      app_prefix: "clove",
      ip_type: :private,
      show_debug: false
    ]
  ]

I then pass in aws:cloudformation:stack-id as the ec2 tag name, which is the same for all ec2 instances in the cluster.

Now, there’s an important caveat that I use 1-1 mapping between ec2 instance and running application instance. So, I’m assuming that this will work if it’s 1-many, but I can’t guarantee that.

See ClusterEC2.Strategy.Tags — libcluster_ec2 v0.6.0 for the strategy

3 Likes
3

Thank you!

5

Can you elaborate a bit on this part? Do you simply set "ec2_tagname": "aws:cloudformation:stack-id" in your Task Definition or is this a value you fetch from elsewhere?

I’m also more familiar with a different hosting option and picking up AWS as needed.

6

Yep! I don’t have access to this anymore, but I’m 99% sure I just set the ec2_tagname like that. The library will look for all nodes that have the same tag value. Because all nodes in the stack have the same stack ID, that tag is useful for discovery.

1 Like
7

Do you know which libcluster strategy support on lightsail AWS ?

8

Hmm… I’ve confirmed that the tag aws:cloudformation:stack-id does exist but the server nodes aren’t able to discover each other with the following:

config :libcluster,
  topologies: [
    example: [
      strategy: ClusterEC2.Strategy.Tags,
      config: [
        ec2_tagname: "aws:cloudformation:stack-id",
        app_prefix: "app"
      ]
    ]
  ]

Libcluster itself is working fine as everything works locally with the Epmd strategy. Anyone else have a working setup for libcluster_ec2 using cloud formation stack ID?

9

Here’s what my config was:

config :my_app_name, :cluster_topology,
  main: [
    strategy: ClusterEC2.Strategy.Tags,
    config: [
      ec2_tagname: "aws:cloudformation:stack-id",
      # Should match what's in vm.args / distillery
      app_prefix: "my_app_name",
      ip_type: :private,
      show_debug: false
    ]
  ]

Make sure the app_prefix matches your vm.args file. It looks like ip_type defaults to private so that isn’t necessary.

If you can’t connect still, then see if you can make a node-to-node connection using Node.connect. If you can’t, that means the BEAM cookie is invalid, your app name is wrong, or the nodes can’t communicate to each other for some other reason (like security groups).

1 Like