AMQP client and TLSv1.3 / OTP23

n1c0

Hello,

I am trying to connect the AMQP client to RabbitMQ server over TLSv1.3 and I am facing the following issue:

iex> AMQP.Connection.open([host: "playground.mydomain.net", port: "10001", username: "toto", password: "tata", ssl_options: [versions: [:"tlsv1.3"]]])

[12:36:38.655] - []
[supervisor: {#PID<0.707.0>, :amqp_connection_sup}, started: [pid: #PID<0.709.0>, name: :connection, mfargs: {:amqp_gen_connection, :start_link, [#PID<0.708.0>, {:amqp_params_network, "toto", {:plaintext, "tata"}, "/", 'playground.mydomain.net', 10001, 0, 0, 10, 50000, [server_name_indication: 'playground.mydomain.net', versions: [:"tlsv1.3"]], [&:amqp_auth_mechanisms.plain/3, &:amqp_auth_mechanisms.amqplain/3], [], []}]}, restart_type: :intrinsic, shutdown: :brutal_kill, child_type: :worker]]

when i try to use TLSv1.2, it works fine

iex> AMQP.Connection.open([host: "playground.mydomain.net", port: "10001", username: "toto", password: "tata", ssl_options: [versions: [:"tlsv1.2"]]])
...
{:ok, %AMQP.Connection{pid: #PID<0.693.0>}}

I am running the following versions for Elixir/Erlang:

Erlang/OTP 23 [erts-11.1] [source] [64-bit] [smp:4:4] [ds:4:4:10] [async-threads:1] [hipe]

Elixir 1.10.4 (compiled with Erlang/OTP 23)

RabbitMQ server TLS is done via a TLS terminating proxy, so i enabled/disabled the TLSv1.2/TLSv1.3 for each attempt, they are not working both at the same time. The certs are Let’s Encrypt issued.

The same termination proxy handles :hackney as an HTTP TLSv1.3 client without any issue.

Any idea on what is going on ?

Thanks for your help

7 comments

#rabbitmq #ssl #tls

1 2939 7

2020-10-06 19:16:18 UTC

Showing Posts 1 to 7

Show Best Posts
Show All (oldest first)
Show All (newest first)

voltone

It’s hard to say, because the error message you are quoting only says that a supervisor repeatedly failed to start the :amqp_gen_connection GenServer. I can’t tell what error, if any, was triggered during the TLS handshake.

What happens when you try to connect with :ssl directly, e.g. :ssl.connect('playground.mydomain.net', 10001, server_name_indication: 'playground.mydomain.net', versions: [:"tlsv1.3"])? That may not be exactly what :amqp_gen_connection is trying to do, as it might set its own additional ssl options, but it might give us a clue…

Post #1

n1c0

Hi Bram, thanks for your quick reply once again!

I manage to connect:

iex> :ssl.connect('playground.mydomain.net', 10001, server_name_indication: 'playground.mydomain.net', versions: [:"tlsv1.3"])
{:ok,
 {:sslsocket, {:gen_tcp, #Port<0.21>, :tls_connection, :undefined},
  [#PID<0.693.0>, #PID<0.692.0>]}}

I tried to look into the AMQP library code a bit and it relies upon the amqp_client Erlang package, i see from the docs The amqp_client application in the amqp_connection module that the ssl_options are the ones from ssl:connect/2 function, so I don’t understand what’s going on…

Post #2

voltone

It probably adds some options of its own. Instead of trying to reverse engineer the options from the code, it might be easier to just trace the call to :ssl.connect:

iex(1)> :dbg.tracer         
{:ok, #PID<0.346.0>}
iex(2)> :dbg.p(:all, :call)
{:ok, [{:matched, :nonode@nohost, 128}]}
iex(3)> :dbg.tp(:ssl, :connect, :cx)
{:ok, [{:matched, :nonode@nohost, 3}, {:saved, :cx}]}
iex(4)> AMQP.Connection.open(<...>)
#...

The output should tell us exactly what ssl options were set, and also whether the handshake succeeds or not. It is possible the TLS handshake is not the issue, but rather something in a higher protocol layer that somehow is impacted by the TLS version.

Post #3

n1c0

Here’s what I get

(<0.720.0>) call ssl:connect(#Port<0.23>,[{versions,[]},{server_name_indication,"playground.mydomain.net"}],50000) ({amqp_network_connection,
                                                                                                                   do_connect,
                                                                                                                   4})
** (exit) exited in: :gen_server.call(#PID<0.720.0>, :connect, 60000)
    ** (EXIT) time out
    (stdlib 3.13.2) gen_server.erl:246: :gen_server.call/3
    (amqp 1.6.0) lib/amqp/connection.ex:158: AMQP.Connection.do_open/2

Post #4

voltone

Ah, so there we have it: {versions,[]}. Looks like the AMQP or amqp_client library filters the version list, and you’re left with an empty list. I don’t have time to go through the code right now, but if you search for “TLSv1.2” in both projects you may find something. Maybe there’s already an update or PR to add it…

Post #5

n1c0

Found why it returns an empty list…the FIXME was never fixed…although we have OTP23 released…

-define(BAD_SSL_PROTOCOL_VERSIONS, [
                                    %% POODLE
                                    sslv3,

                                    %% Client side of TLS 1.3 is not yet
                                    %% implemented in Erlang/OTP 22.0
                                    %% prereleases. As a consequence,
                                    %% not sure about the stability of
                                    %% the server side.
                                    %%
                                    %% FIXME: Revisit this decision when
                                    %% Erlang/OTP 22.0 final release is
                                    %% out.
                                    'tlsv1.3'
                                   ]).

in rabbit_common pkg, in the file rabbit_ssl_options.erl

Post #6

voltone

They had to wait a little longer, I guess: you don’t want to enable TLS 1.3 before :ssl version 10.0, which means OTP 23.0. Even 9.6.2.2, the latest :ssl version on the 22.3 branch, has interop issues.

In Mint we ended up selecting the default versions by checking the supported protocol versions, but then removing TLS 1.3 pre-23 (here). Still, it lets the caller override it and enable any version. Would be nice if rabbit_common would make this filter :ssl app version dependent too.

Post #7

— All posts loaded —