BTW, Partisan doesn’t have support for global yet nor other capabilities like pg but I plan on adding those too. As always the challenge is implementing it correctly and efficiently when using HyParView or any user-defined membership strategy using partial-views/partial-mesh
aramallo
1
chasers
chasers
Also fwiw, in the regions we’re in the Fly network has been a lot better in the last few months.
1
hubertlepicki
I’m interested in that too. I’ve seen similar issues with nodes disconnecting or traffic between them “hanging” from time to time and it’s been quite frustrating. I understand that Erlang distribution wasn’t built for this sort of deployment in mind, and yet people do run huge clusters in the cloud (see WhatsApp) and it somehow gets orchestrated more reliably than what fly.io and (Gigalixir)[Clustering | Gigalixir] docs suggest.
0
D4no0
With the amount of money they have, they might have some specific in-house built tools to deal with these problems.
0
hubertlepicki
AFAIK they used to run a custom Erlang distribution mechanism called “WANDIST” but they gave up on the idea and the cluster as they currently have is a mostly standard Erlang distribution:
Maybe we can summon @max-au here to share some tips on things like tweaking the defaults and general approach to clustering over semi-reliable infrastructure that happens to be the cloud.
But overall, this should be a “solved problem” and not happening on public clouds, especially those that target Elixir/Erlang deployments like Fly.io or Gigalixir and it would be within their best interest to develop and share techniques that we can incorporate to avoid random clustering issues from time to time.
0
hubertlepicki
Generally this wouldn’t be a big issue but most folks here probably would like to use Phoenix PubSub, and it relies either on Redis or pg backends. Do you happen to know a Partisan PubSub backend?
0
