Setting up a cluster on Docker cloud (with docker swaarm)

Edit

  • I have created this project template for starting a clustered application using Docker.

I have been trying to work out to establish and erlang cluster on docker cloud.

There are instructions for kubernetes. But I cannot find any instructions for swarm.

Does anyone have instructions for doing this or an example project?

Questions I include

  • Does the port for EPMD need to be explicitly opened?
  • Can nodes have multiple longnames so you can connect via myapp@service or myapp@service-1
  • How do dynamically start a node so it is given the correct name?
  • How can an iex session be attached to an elixir/erlang node running in a docker swarm
  • Is there a way to build with code reloading and have multiple running.
  • If not do you develop with compose then run swarm locally and surface test?

Answer

Swarm/Cloud

At start up each node can try to connect to the first node in the service (or first n for redundancy).
To be able to connect using host names the machines must be using --sname option.

Start container with

elixir --sname app --cookie secret -S mix run --no-halt

Within the application start block connect to first node.

Node.connect(:"app@www-1")
Node.connect(:"app@www-2")
Node.connect(:"app@www-3")

Docker compose

A random identifier is the default host when starting with --sname so will not start a node that can be connected to

Notes

  • Docker swarm adds to the docker ecosystem the concept of services. A service is several containers all running the same version of an image. Given a service called www with three nodes the hostname www will dynamically resolve to one of the nodes. The host names www-1, www-2 and www-3 will resolve to specific containers
  • In docker swarm ports seam to be open by default, there is no requirement to explicitly open the epmd port.
    I think the fact the epmd port is used should be declared through the uses of exposes in the Dockerfile
  • erlang used hostname -f for find it’s hostname if given an -sname on cloud this is like the service name on compose it looks like a random identifier
1 Like

The purpose of containers is to have static code inside image and do not change anything in it.
If you need new version to deploy, you build new image and let know docker swarm you want to upgrade version in service.

Also nice to have health check implemetned
https://blog.newrelic.com/2016/08/24/docker-health-check-instruction/

Ports are only open in the same “network”, you can create and specify on which network service is running.
Example on more advanced application

I don’t believe that to be the purpose of containers. True I think immutable infrastructure and containers are both a good idea but neither requires the other. I want to have static code in the images I push to production, but in development I want edits to be reflected in the running system.

The comment you highlighted I made in reference to a swarm running on my dev machine. Or possibly a container running using docker compose on my dev machine. Which ever works best really.

What I prefer for development docker compose, for production docker swarm (files for docker compose, docker swarm are similar and easy to share between ). For development you can have mounted local folder into container with code, or just build container every time you made changed in good as you prefer.

Yep that is exactly what I do. I just haven’t worked out how to do clustering in compose. The problem is the container hostname in compose is not accessible on the network so erlang clustering is a pain.

I need clustering locally, because I have test cases that need to check consensus has been reached

OK I I think get it, you want to auto scale swarm service adding nodes, and you want that erlang new node will be automatic connected with other nodes.

Something like distributed cache https://github.com/bitsofinfo/hazelcast-docker-swarm-discovery-spi

This works with Hazelcast 3.6-EA+ and Docker Swarm services (manager nodes) to provide automatic peer discovery of the Hazelcast cluster. This is done by interrogating the swarm manager api’s to locate peers based on shared information such as network info, service names and service labels (that are specified by you when you launch your services)

https://docs.docker.com/docker-cloud/apps/service-links/

Also maybe this will help

Thanks those resources were all helpful. My efforts so far are summarised in this repo

4 Likes