mindreader

mindreader

Classification of a nodes in a cluster?

We currently have a monolith where I work, on kubernetes. We are using libcluster to get the nodes the mesh together, and up until now, there has only been one type of node that has every possible dependency and library. It isn’t really sustainable so I’d like to break up some functionality.

The way libcluster works is it gets a list of ips where erlang nodes could be, and then connects them together. Since there could be multiple erlang vms on one machine, it uses a basename@hostname combo for its long names, where the hostname determines what machine it is on, and the basename determines which vm on that machine you want.

In kubernetes every container is just going to have one node on it, so libcluster made a design decision that every erlang node should have the same basename. That’s fine, but then how do I tell the difference between node types in my cluster?

For example, if I have some api nodes, and I have some analytics nodes, and one of the api nodes wants to contact a random analytics node to crunch some data and wait for the results, how do I determine which nodes are “analytics” nodes, when all of the nodes names look like :foo@whatever ?

Up until now I would have just named them api@whatever and analytics@whatever and just done a Node.list and filtered out the analytics nodes, but I don’t see a lot of people doing that so it makes me wonder if there is a better way.

Most Liked

shanesveller

shanesveller

Most of the time we’d use Pod labels and a ClusterIP or headless Service that selects over those labels to do sibling discovery, which is a model libcluster directly supports. In many environments you don’t have any direct connectivity to etcd that powers the cluster, for security and safety reasons, and some don’t actually use etcd under the hood. IIRC Amazon EKS is actually persisting to DynamoDB or something to that effect.

chasers

chasers

I think a lot of k8s people use a service registry like etcd.

I would be interested in another way however as I’m choosing not to go down the k8s route (for now … using Google’s instance groups) but I’m at a place where I need to deregister a node when it receives a sigterm. Perhaps Swarm or Syn.

Where Next?

Popular in Questions Top

New
lastday4you
I wanted to check elixir version in phoenix because i found that my elixir is 1.5 but when i use Enum.chunk_by it said the function is un...
New
baxterw3b
Hi guys, i’m new in the Elixir world, and i have to say, that i love it! i’m having some problem to understand anonymous functions with ...
New
shahryarjb
Hello, I have map which I want to convert it to string like this: the map: %{last_name: "tavakkoli", name: "shahryar"} the string I ne...
New
belgoros
I’m not a pro in using Regex and can’t figure out why the following behaviour happens, especially if we take into account the difference ...
New
Lily
In templates/appointment/index.html.eex: <%= for appointment <- @appointments do %> <tr> <td><%= appoi...
New
itssasanka
Hi all, Trying to get some more clarity over utc_datetime and naive_datetime for Ecto: The documentation above suggests that while ...
New
jay1
Why is it that the mnesia database isn’t the most preferred database for use in Elixir/Phoenix?
New
sergio_101
I am VERY much an elixir newbie. I have taken one elixir course and one phoenix course on Udemy. During that course, I saw the instructor...
New
svb
Hi! Currently I want to submit a form by pressing the Enter key. However, since my input field is of type “textarea” this is just adds a...
New

Other popular topics Top

aadeshere1
I have a another noob question about loop. Since elixir is immutable, while loop is not directly possible. total = 10 while total != 0 ...
New
WestKeys
Currently suffering from paralysis by [HTTP client] analysis. This is rather unusual in Elixirland as there tends to be consensus on the ...
New
shahryarjb
Hello, I have map which I want to convert it to string like this: the map: %{last_name: "tavakkoli", name: "shahryar"} the string I ne...
New
jerry
Good day to you all. I have been struggling to get a query involving like and ilike to work. Can anyone assist me on this, please? pro...
New
stefanchrobot
What’s the safe way to decode a JSON string into a struct? I want to avoid calling String.to_atom. Jason.decode can give me a map with st...
New
jay1
Why is it that the mnesia database isn’t the most preferred database for use in Elixir/Phoenix?
New
Emily
I have VueJS GUIs with the project generated using Webpack. I have Elixir modules that will need to be used by the VueJS GUIs. I forese...
New
grych
Hi folks, Few months ago I have announced the proof-of-concept of the library to manipulate the browsers DOM objects directly from Elixi...
639 52673 488
New
saif
Hello everyone, Long time lurker first time poster here. I’ve recently begun working on Elixir full-time again! :raised_hands: It’s been...
New
AstonJ
Seen any cool LiveView demos, sample apps or examples? Please post them here! :003:
New

Latest on Elixir Forum

Elixir Forum

We're in Beta

About us Mission Statement