Libcluster Postgres - Clustering Strategy for Libcluster

ziinc

Hi, my colleagues and I at Supabase have worked on libcluster_postgres, a libcluster custom strategy that utilizes PostgreSQL LISTEN and NOTIFY to handle dynamic clustering. This library came out of a requirement for a more reliable discovery method that was cloud agnostic.

By default, the node’s cookie is used as the channel identifier, but you can customize it to anything you want, so long as it is a valid PostgreSQL identifier.

This clustering strategy is being used across 3 different products written in Elixir - Realtime, Supavisor, and Logflare.

@filipecabaco @wenbo @chasers @abc3

8 comments

#library #deployment #postgrex #libraries #libcluster

43 2051 8

2025-07-02 18:28:11 UTC

Showing Posts 1 to 8

Show Best Posts
Show All (oldest first)
Show All (newest first)

jimsynz

Ash Core Team

This is great work.

I’ve used Redis before to do this job, but since there’s almost zero chance I’ll have to build a clustered system that doesn’t contain postgres I really like this solution.

Congrats!

Post #1

beepbeepbopbop

I’ve given this a go, seems to work wonders. That being said, I think the example in the document doesn’t seem right:

config :libcluster,
  topologies: [
    example: [
      strategy: Cluster.Strategy.Postgres,
      config: [
          hostname: "localhost",
          username: "postgres",
          password: "postgres",
          database: "postgres",
          port: 5432,
          parameters: [],
          # optional, defaults to node cookie
          channel_name: "cluster"
      ],
    ]
  ]

Shouldn’t this be LibclusterPostgres.Strategy based from this? libcluster_postgres/lib/strategy.ex at main · supabase/libcluster_postgres · GitHub

Post #2

filipecabaco

you are right! this was from an earlier iteration

will push a fix to the docs

update: fix: Change URLs in Readme and fix example by filipecabaco · Pull Request #13 · supabase/libcluster_postgres · GitHub

Post #3

ziinc

v0.1.2 Release

Link to full changelog

Support for ssl and ssl_opts connection options by @joshknz in #14 for custom SSL certs.

Post #4

acrolink

@ziinc

Thanks for the great library. I am trying to make it work inside an AWS Elastic Beanstalk environment, and using a RDS PostgreSQL 16 server.

Dockerfile specifications:

ARG ELIXIR_VERSION=1.15.7
ARG OTP_VERSION=26.2.5
ARG DEBIAN_VERSION=bullseye-20240513-slim

One instance reports, as an example:

21:00:14.839 [error] [libcluster:example] Failed to connect to node: myapp@c72329f86177
21:00:19.847 [warning] [libcluster:example] unable to connect to :myapp@c72329f86177

What can be the reason for that?

Post #5

samrose

@acrolink it’s probably an aws networking issue. The error is elixir reporting that nodes in the BEAM network are failing to connect as far as I can tell.

Post #6

tfwright

Just wanted to leave a note and say, I was having a ton of minor headaches relying on epmd, this just worked. Just in time for needed pubsub/presence features. Thanks for sharing a great library!

Post #7