Connect Livebook to a Elixir node running in a Docker container

Wigny

Hey folks, I would like help to connect Livebook to an Elixir node running in a Docker container.

To run the container I usually do

docker-compose run --rm --service-ports api iex -S mix

and Livebook

docker run -p 8080:8080 -p 8081:8081 --pull always -u $(id -u):$(id -g) -v $(pwd):/data ghcr.io/livebook-dev/livebook:nightly

as described in the docs.

Now I want to attach a new notebook to the running app. If I got it right, for the node to be visible to Livebook, it must be running in the same network and have a known RELEASE_NODE and RELEASE_COOKIE. For that, I tried following this gist, this thread and also this discussion, but got no luck.

This is what I tried:

docker-compose run \
  --rm \
  --service-ports \
  -e RELEASE_DISTRIBUTION="name" \
  -e RELEASE_NODE="api@127.0.0.1" \
  -e RELEASE_COOKIE="mycookie" \
  api \
  iex --name api@127.0.0.1 --cookie mycookie -S mix

docker run \
  -p 8080:8080 \
  -p 8081:8081 \
  --pull always \
  -u $(id -u):$(id -g) \
  -v $(pwd):/data \
  --network $API_NETWORK \
  -e LIVEBOOK_NODE="livebook@127.0.0.1" \
  -e LIVEBOOK_DISTRIBUTION=name \
  -e LIVEBOOK_DEFAULT_RUNTIME="attached:api@127.0.0.1:mycookie" \
  ghcr.io/livebook-dev/livebook:nightly

What am I missing?

8 comments

/livebook #docker

1 764 8

2025-02-22 05:32:16 UTC

Showing Posts 1 to 8

Show Best Posts
Show All (oldest first)
Show All (newest first)

ruslandoga

@Wigny

I think 127.0.0.1 might be the issue (Docker typically uses 172.x.x.x/16 subnets). Have you tried using container names for hostnames?

Post #1

jonatanklosko

Creator of Livebook

Hey @Wigny, I think @ruslandoga is correct, the iex node needs to be api@[CONTAINER_NAME], as mentioned in the gist.

Sidenote:

you don’t need the RELEASE_* env vars, they only apply to releases, but not for iex -S mix (and you already set --name and --cookie instead)
LIVEBOOK_DISTRIBUTION=name no longer has any effect, so you can skip it

Post #2

arcanemachine

Because of how they’re exposing the ports 8080:8080, I think this will expose them like any other service on the host (i.e. nmap localhost would show port 8080 as being open). If the syntax was just 8080, then I believe your assumption would be correct. (Might be worth a shot also, plus better to isolate the containerized service in its own Docker network… Just gotta make sure the container name is correct then.)

Also could

Post #3

ruslandoga

Interesting, I’ll try it out.

Post #4

ruslandoga

I wasn't able to access another container in the same bridge network via localhost.

$ docker network create livebook
4c530174a62315fa9e7653147e8b3d2fbb7a16db089abf00b6600b14269c72ac

$ docker network inspect livebook
[
    {
        "Name": "livebook",
        "Id": "4c530174a62315fa9e7653147e8b3d2fbb7a16db089abf00b6600b14269c72ac",
        "Created": "2025-02-16T10:39:41.965962798Z",
        "Scope": "local",
        "Driver": "bridge",
        "EnableIPv6": false,
        "IPAM": {
            "Driver": "default",
            "Options": {},
            "Config": [
                {
                    "Subnet": "172.18.0.0/16",
                    "Gateway": "172.18.0.1"
                }
            ]
        },
        "Internal": false,
        "Attachable": false,
        "Ingress": false,
        "ConfigFrom": {
            "Network": ""
        },
        "ConfigOnly": false,
        "Containers": {},
        "Options": {},
        "Labels": {}
    }
]

$ docker run -p 80:80 -d --network livebook nginx:alpine
aaa54c3c62e7bd1c80d2db3a76064a9f60dda209b6beb5fab7aa1637930277f6

$ curl http://localhost:80
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
html { color-scheme: light dark; }
body { width: 35em; margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif; }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>

<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>

<p><em>Thank you for using nginx.</em></p>
</body>
</html>

$ docker network inspect livebook
[
    {
        "Name": "livebook",
        "Id": "4c530174a62315fa9e7653147e8b3d2fbb7a16db089abf00b6600b14269c72ac",
        "Created": "2025-02-16T10:39:41.965962798Z",
        "Scope": "local",
        "Driver": "bridge",
        "EnableIPv6": false,
        "IPAM": {
            "Driver": "default",
            "Options": {},
            "Config": [
                {
                    "Subnet": "172.18.0.0/16",
                    "Gateway": "172.18.0.1"
                }
            ]
        },
        "Internal": false,
        "Attachable": false,
        "Ingress": false,
        "ConfigFrom": {
            "Network": ""
        },
        "ConfigOnly": false,
        "Containers": {
            "aaa54c3c62e7bd1c80d2db3a76064a9f60dda209b6beb5fab7aa1637930277f6": {
                "Name": "strange_kalam",
                "EndpointID": "0193b7811de9e139f7f5a562c789d332d409241c3d7a1be89b9dcf14ffaba133",
                "MacAddress": "02:42:ac:12:00:02",
                "IPv4Address": "172.18.0.2/16",
                "IPv6Address": ""
            }
        },
        "Options": {},
        "Labels": {}
    }
]

$ docker run -ti --network livebook alpine

# apk add curl

# curl http://localhost:80
curl: (7) Failed to connect to localhost port 80 after 0 ms: Could not connect to server

# ip addr show
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: tunl0@NONE: <NOARP> mtu 1480 qdisc noop state DOWN qlen 1000
    link/ipip 0.0.0.0 brd 0.0.0.0
3: gre0@NONE: <NOARP> mtu 1476 qdisc noop state DOWN qlen 1000
    link/gre 0.0.0.0 brd 0.0.0.0
4: gretap0@NONE: <BROADCAST,MULTICAST> mtu 1462 qdisc noop state DOWN qlen 1000
    link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff
5: erspan0@NONE: <BROADCAST,MULTICAST> mtu 1450 qdisc noop state DOWN qlen 1000
    link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff
6: ip_vti0@NONE: <NOARP> mtu 1480 qdisc noop state DOWN qlen 1000
    link/ipip 0.0.0.0 brd 0.0.0.0
7: ip6_vti0@NONE: <NOARP> mtu 1428 qdisc noop state DOWN qlen 1000
    link/tunnel6 00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00 brd 00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00
8: sit0@NONE: <NOARP> mtu 1480 qdisc noop state DOWN qlen 1000
    link/sit 0.0.0.0 brd 0.0.0.0
9: ip6tnl0@NONE: <NOARP> mtu 1452 qdisc noop state DOWN qlen 1000
    link/tunnel6 00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00 brd 00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00
10: ip6gre0@NONE: <NOARP> mtu 1448 qdisc noop state DOWN qlen 1000
    link/[823] 00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00 brd 00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00
26: eth0@if27: <BROADCAST,MULTICAST,UP,LOWER_UP,M-DOWN> mtu 1500 qdisc noqueue state UP
    link/ether 02:42:ac:12:00:03 brd ff:ff:ff:ff:ff:ff
    inet 172.18.0.3/16 brd 172.18.255.255 scope global eth0
       valid_lft forever preferred_lft forever

# curl http://172.18.0.2
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
html { color-scheme: light dark; }
body { width: 35em; margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif; }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>

<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>

<p><em>Thank you for using nginx.</em></p>
</body>
</html>

Would you be able to modify my example to make it work?

Post #5

Wigny

Thanks for the tips!

So I got rid of the unnecessary env vars and started giving a name to my container:

docker-compose run \
 --rm \
 --service-ports \
 --name api \
 api \
 iex --name test@api --cookie mycookie -S mix phx.server

docker run \
 -p 8080:8080 \
 -p 8081:8081 \
 --pull always \
 -u $(id -u):$(id -g) \
 -v $(pwd):/data \
 --network $API_NETWORK \
 -e LIVEBOOK_DEFAULT_RUNTIME="attached:test@api:mycookie" \
 ghcr.io/livebook-dev/livebook:nightly

but when trying to attach Livebook, I’m getting in the logs ** Hostname api is illegal **.

Given our api app has a /api/ping endpoint, I tried checking that within the Livebook container we can access the endpoint using wget. So by attaching to the Livebook container running docker exec -it $CONTAINERID bash and executing wget -qO- localhost:4000/api/ping or wget -qO- api:4000/api/ping I had no response, but wget -qO- host.docker.internal:4000/api/ping or wget -qO- 172.17.0.1:4000/api/ping returns pong…

Post #6

jonatanklosko

Creator of Livebook

** Hostname api is illegal **

Ah, when running using long names by specifying --name (which Livebook now always does as well), if the hostname needs to be either an IP or a fully qualified domain name (basically, it needs at least one dot). Looks like it’s only validated when trying to connect to another node, not when starting the node itself.

One way would be for the container name to include a dot, but I believe that can also just use {container_name}.{network_name} as the hostname and the internal Docker DNS will resolve that correctly.

Post #7

Wigny

That worked like a charm! Many thanks!

Leaving the full command here for future readers:

docker-compose run \
 --rm \
 --service-ports \
 --name api \
 api \
 iex --name test@api.$API_NETWORK --cookie mycookie -S mix

docker run \
 -p 8080:8080 \
 -p 8081:8081 \
 --pull always \
 -u $(id -u):$(id -g) \
 -v $(pwd):/data \
 --network $API_NETWORK \
 -e LIVEBOOK_DEFAULT_RUNTIME="attached:test@api.$API_NETWORK:mycookie" \
 ghcr.io/livebook-dev/livebook:nightly

Post #8

— All posts loaded —