Cannot set port range for node-node connections (-kernel inet_dist_listen_min/max)?

Questions / Help
1

I have spent the past day trying to limit my port range for node-node connections.

Grok spent hours on it with me eventually concluding the documentation is too sparse and the mechanisms too opaque. It wanted me to start dissecting erlang and elixir as it ran out of ideas.

ChatGPT was a lost cause (as I have said before, if the largest AI models cannot figure out an API…)

I found one blog post of someone who said they did it:

I tried numerous methods. I ran mix release.init to make a /rel folder.

Then I tried vm.args.eex:

# DOESN'T WORK, CAN'T INSERT VARIABLE HERE (not sh)
#-kernel inet_dist_listen_min ${ERL_DIST_PORT_MIN}
#-kernel inet_dist_listen_max ${ERL_DIST_PORT_MAX}

# WORKS BUT REQUIRES MANUAL PERMANENT STATE
-kernel inet_dist_listen_min 9100
-kernel inet_dist_listen_max 9200

I tried commenting that out and just doing it by arguments:

    /app/bin/main_server start_iex \
        --erl "-kernel inet_dist_listen_min $ERL_DIST_PORT_MIN" \
        --erl "-kernel inet_dist_listen_max $ERL_DIST_PORT_MAX" \
        --erl "+Q $MAX_SOCKETS"

or

/app/bin/main_server start_iex \
        --erl "-kernel inet_dist_listen_min 9000" \
        --erl "-kernel inet_dist_listen_max 9200" \
        --erl "+Q $MAX_SOCKETS"

or

/app/bin/main_server start_iex \
        --erl "-kernel inet_dist_listen_min 9000 inet_dist_listen_max 9200 +Q $MAX_SOCKETS"

These do nothing though.

So it seems:

  • The --erl kernel arguments are ignored (Why?)
  • vm.args.eex can’t take any environmental arguments that will be parsed at runtime from environment (or can it?)
  • Thus our only method is to hardcode into vm.args.eex?

Here is some debug out when just using the --erl like recommended in the other article

iex(websock01@websock01)4> System.cmd("epmd", ["-names"]) # should be 9100
{"epmd: up and running on port 4369 with data:\nname websock01 at port 35647\n",
 0}
iex(websock01@websock01)5> System.get_env("ERL_DIST_PORT_MIN") # "9100"
"9100"
iex(websock01@websock01)6> System.get_env("ERL_DIST_PORT_MAX") # "9200"
"9200"
iex(websock01@websock01)7> :init.get_arguments() # confirm release mode
[
  root: [~c"/app"],
  bindir: [~c"/app/erts-15.2.1/bin"],
  progname: [~c"erl"],
  home: [~c"/app"],
  noshell: [],
  elixir: [~c"ansi_enabled", ~c"true"],
  user: [~c"elixir"],
  mode: [~c"embedded"],
  setcookie: [~c"mysecretcookie"],
  sname: [~c"websock01"],
  config: [~c"/app/releases/0.1.0/sys"],
  boot: [~c"/app/releases/0.1.0/start"],
  boot_var: [~c"RELEASE_LIB", ~c"/app/lib"]
]
iex(websock01@websock01)8> System.get_env("NODE_NAME") # "websock01"
"websock01"
iex(websock01@websock01)9> :erlang.system_info(:port_limit) # reflective of MAX_SOCKETS (set to +Q in erlang of start_iex --erl)
1048576

Wheres when they are hardcoded into the vm.args.eex it works and i see the kernel flags in get_arguments.

Is there any dynamic system that works that would let me change these ports without rebuilding the whole docker image and is there any reason --erl doesn’t work?

Can I do system.get_env maybe in the vm.args.eex or will that only be parsed at build or not be permitted either?

Thanks.

2

Furthermore, while the vm.args files are static, you can use env.sh and env.bat to dynamically set VM options. For example, if you want to make sure the Erlang Distribution listens only on a given port known at runtime, you can set the following:

case $RELEASE_COMMAND in
  start*|daemon*)
    ELIXIR_ERL_OPTIONS="-kernel inet_dist_listen_min $BEAM_PORT inet_dist_listen_max $BEAM_PORT"
    export ELIXIR_ERL_OPTIONS
    ;;
  *)
    ;;
esac

https://hexdocs.pm/mix/Mix.Tasks.Release.html#module-vm-args-and-env-sh-env-bat

3

:wave:

I think if you enable “restart after loading and evaluating config/runtime.exs” via reboot_system_after_config you would be able to specify :kernel configuration there.

# mix.exs
# ...
def project do
  [
    # ...
    releases: [
      my_app: [
        # ...
        reboot_system_after_config: true
      ]
    ]
  ]
end

# config/runtime.exs
# ...
config :kernel,
  inet_dist_listen_min: String.to_integer(System.get_env("BEAM_PORT")),
  inet_dist_listen_max: String.to_integer(System.get_env("BEAM_PORT")),