My app does not pick up the changed hostname on my VPS

Hi everyone.

I’m trying to setup a small cluster with three nodes. I’m using libcluster as I have before. But my nodes doesn’t have a usable hostname. The VPS came with a hostname like this: ip-123-123-123-123. I’ve changed the hostname on the VPS but it doesn’t get picked up in the app when I restart it. I’ve restarted the server. And i can see that the hostname has changed on the server it self.

I’m seeing in the logs when it starts that it’s using the old hostname, which included the IP of the vps. So I get this error:

21:36:48.538 [error] ** System running to use fully qualified hostnames **
: ** Hostname ip-x-x-x-x is illegal **

The app is built using Mix release. From the docs I can see that I should be able to set RELEASE_DISTRIBUTION and RELEASE_NODE env variables. I’ve set them to the following:

RELEASE_DISTRIBUTION=name
RELEASE_NODE=app_name@host.url.com

I would assume doing so would force change the hostname in the app as well. But it doesn’t seem to be the case. What am I missing. Thanks in advance :slightly_smiling_face:

This is what I found when looking for what makes an hostname valid as opposed to illegal. Maybe you’re missing the resolvable and reachable part.

Long names need the hostname to be resolvable and reachable, in other words it needs to either be an IP address or a valid DNS name. In the case of an IP address, it is already resolved so it only needs to be reachable from the current host. In the case of a DNS name, it needs to be resolvable by any one of the DNS resolution mechanisms on the host (/etc/hosts, /etc/resolve.conf, etc.) to an IP address which is reachable from the current host.

1 Like

Thanks for your answer.

The hostname is indeed resolvable and reachable. But it doesn’t seem to pick up the hostname at all in my app. That’s the problem I think.

It keeps using the original hostname the VPS had when it was first setup. Now the hostname has changed to something that is reachable from the outside world as well. But it doesn’t seem to pick up the host name even though I set it explicitly with tne RELEASE_NODE env variable.

Where and how did you change those?

I changed the hostname in /etc/hostname and rebooted the server.

The RELEASE_NODE & RELEASE_DISTRIBUTION env variables is set in the systemd service responsible for keeping the app running

If the output of both hostname and hostname -f are not the same then I think your hostname is not set properly.

Try to set it with:

sudo hostnamectl set-hostname something.example.com

Thanks for your input.

The output of both hostname and hostname -f are the same.

I tried setting the hostname with your command. Sadly it didn’t make any difference.

Maybe moving the Elixir configuration for http and url and anything not related with compile time settings into runtime.exs and build a new release can help?

Yeah I already did that and that works fine. But the node it self doesn’t get the correct host.

So, how are you setting up the nodes configuration?

So, how are you setting up the nodes configuration?

What exactly do you mean by nodes configuration? Then only thing I’m trying now is to make sure my node has the right hostname (using RELEASE_NODE & RELEASE_DISTRIBUTION ) so I can connect to it from another node. After that I’m setting up libcluster.

If anyone think they can help and would like to pair with me on this it would be much appreciated. I would of course pay for the time spend even if we don’t get it working.

Currently the info you have given seems to broad for the ones with experience in libcluster and releases may try to help you.

Try to make a simple repo where the problem is reproducible and then share the link to it here.

Also mention what hosting provider you are using and if you are using any special deployment strategy.

I’m using AWS Lightsail instances.

Libcluster is not a part of the problem right now. It’s just getting rid of the error caused by the node getting the old hostname. I’ll try setting up a new instance and see if that fixes the issue.

This is a bit grasping for straws here, but are you sure the correct environment variable values reach the release startup scripts (and the application)? I’ve set up libcluster successfully in AWS Fargate using RELEASE_DISTRIBUTION and RELEASE_NODE and this was one of the hurdles. Unfortunately I don’t know Lightsail that much.

To debug this you can try to sprinkle echo statements in shell scripts and perhaps check the app environment if you have access to the Elixir shell.

Edit: not a bit for grasping straws, more a lot…

I really appreciate your input. Thank you so much for taking the time to grasp for straws.

So I’ve looked a bit more into it. From what I can see, when I look at the process systemctl starts up using the environment variables, it seems to set the correct --name param to the domain that is reachable from the outside world.

But still, the app complains that another Hostname is set, which is not legal. I can’t connect to the running app because it never finishes starting.

I’m running Elixir 1.10.3 and erlang 23.0.2.

Isn’t --sname the correct option to use when using full qualified names?

No, the s in sname is for “short”. Some check in the BEAM will even error if there is a dot in a sname.

1 Like

More straw grasping - you can try to put this at the top/start of some/any scripts you would like to debug:

set -x

it echoes results of the commands in the script file as it executes…

( you can also run it as:
sh -x scriptname
if you can’t/don’t want to edit the scripts themselves)

Best of luck.

1 Like