resrc
November 21, 2022, 3:37pm
1
Our app (phoenix 1.5.14) sends the following response back to mux.com in our webhook controller:
TLS client: In state connection at tls_record_1_3.erl:337 generated CLIENT ALERT: Fatal - Bad Record MAC\n decryption_failed
Found the following related issue, but we’re already on OTP25 (elixir:1.14.2-alpine).
opened 02:27AM - 18 Oct 22 UTC
closed 10:52AM - 18 Oct 22 UTC
bug
**Describe the bug**
TLS connection from erlang 24.1.6 to erlang 24.3.4.5 gets… established and immediately gets closed with the following error in log: `{unexpected_msg,{internal,{change_cipher_spec,<<1>>}}}`
24.3.4.5 -> 24.1.6, 24.1.6 -> 24.1.6, 24.3.4.5 -> 24.3.4.5 connections seem to work fine.
**To Reproduce**
24.3.4.5 node (server):
```
5> {ok, LS} = ssl:listen(21176, [{keyfile, "./pkey.pem"}, {certfile, "./chain.pem"}, {versions, ['tlsv1.3']}]), Accept = fun (LSS) -> {ok, HS} = ssl:transport_accept(LSS, 60000), ssl:handshake(HS, 30000) end, {ok, S} = Accept(LS).
=NOTICE REPORT==== 17-Oct-2022::19:24:39.974949 ===
TLS server: In state wait_finished at ssl_gen_statem.erl:736 generated SERVER ALERT: Fatal - Unexpected Message
- {unexpected_msg,{internal,{change_cipher_spec,<<1>>}}}
** exception error: no match of right hand side value {error,
{tls_alert,
{unexpected_message,
"TLS server: In state wait_finished at ssl_gen_statem.erl:736 generated SERVER ALERT: Fatal - Unexpected Message\n {unexpected_msg,{internal,{change_cipher_spec,<<1>>}}}"}}}
```
24.1.6 node (client):
```
7> ssl:connect({127,0,0,1}, 21176, []).
=WARNING REPORT==== 17-Oct-2022::19:04:55.480291 ===
Description: "Authenticity is not established by certificate path validation"
Reason: "Option {verify, verify_peer} and cacertfile/cacerts is missing"
{ok,{sslsocket,{gen_tcp,#Port<0.9>,tls_connection,undefined},
[<0.135.0>,<0.134.0>]}}
8> =NOTICE REPORT==== 17-Oct-2022::19:04:55.484117 ===
TLS client: In state connection at tls_record_1_3.erl:340 generated CLIENT ALERT: Fatal - Bad Record MAC
- decryption_failed
```
**Expected behavior**
Connections are not expected to be closed.
**Affected versions**
24.3.4.5
**Additional context**
Please see the traffic recording for more context:
[unexpected_msg_tcpdump.zip](https://github.com/erlang/otp/files/9805823/unexpected_msg_tcpdump.zip)
Will appreciate any help, thanks!
This started when crypto .hmac /3 which was removed in OTP 24, so we upgraded to OTP25:
Phoenix 1.5.14
Docker:
elixir:1.14.2-alpine
alpine:3.16.3
compiled app is running erts-13.1.2
Any ideas? Thanks,
1 Like
resrc
November 25, 2022, 11:37am
2
Here’s a short update. No solution as of yet.
When using cURL the issue above does not appear.
curl -d "@data.json" -H "Content-Type: application/json" -X POST https://app.domain.com/.well-known/mux
Our SSL cert looks to be fine and valid.
Could this be a firewall/network issue?
jpunie
November 25, 2022, 12:54pm
3
We have a similar issue.
After upgrading OS (openssl 1.1 to openssl 3), Erlang OTP 25.1.2 and Elixir 1.14.2 on our host. We eventually had to revert back to Erlang 24.3.4 and Elixir 1.13.4.
In our case this was the only option, as our nerves devices in the field could not connect.
Hopefully this can be solved via some kind of easy fix. But probably the encryption setup is just different in the new version and incompatible with older versions?
I’m also hoping someone can explain the error in more detail? What is causing it.
Our install is using OpenSSL 3.0.2.
When installing Erlang 24.3.4 with asdf it provided following info:
crypto : Using OpenSSL 3.0 is not yet recommended for production code.
Everything seems to be working for now.