thousandsofthem

Issue with http clients and some https certificates

Please help me debug the issue with https certificates, need some ideas what to check next

When fetching data using http clients, some https:// urls returning “Certificate Expired”, despite being perfectly valid, e.g. https://investing.com/, https://api.amplitude.com/ .
It looks like it doesn’t depend on erlang version, elixir version, http library version.

Base system is (a) OSX 10.14.5 (b) Docker, Alpine Linux

test app #1

elixir 1.9.4
erlang 22.3.4.1 (also 22.1.8)
mojito 0.6.4
mint 1.1.0
httpoison 1.5.0
hackney 1.15.2 (also 1.16.0)

iex(8)> Mojito.request(:get, "https://api.amplitude.com/")
[info] TLS :client: In state :certify at ssl_handshake.erl:1764 generated CLIENT ALERT: Fatal - Certificate Expired

{:error,
 %Mojito.Error{
   message: nil,
   reason: %Mint.TransportError{
     reason: {:tls_alert,
      {:certificate_expired,
       'TLS client: In state certify at ssl_handshake.erl:1764 generated CLIENT ALERT: Fatal - Certificate Expired\n'}}
   }
 }}

iex(9)> HTTPoison.get("https://api.amplitude.com/")
[info] TLS :client: In state :certify at ssl_handshake.erl:1764 generated CLIENT ALERT: Fatal - Certificate Expired

{:error,
 %HTTPoison.Error{
   id: nil,
   reason: {:tls_alert,
    {:certificate_expired,
     'TLS client: In state certify at ssl_handshake.erl:1764 generated CLIENT ALERT: Fatal - Certificate Expired\n'}}
 }}


iex(10)> opts = [ssl: [cacertfile: :certifi.cacertfile(), verify: :verify_peer]]
iex(11)> HTTPoison.get("https://api.amplitude.com/", [], opts)
# same error - CLIENT ALERT: Fatal - Certificate Expired

test app #2

elixir 1.3.4
erlang 19.3.6.5
hackney 1.15.0
httpoison 1.0.0

iex(6)> HTTPoison.get("https://api.amplitude.com/")
[error] SSL: :certify: ssl_handshake.erl:1609:Fatal error: certificate expired

{:error, %HTTPoison.Error{id: nil, reason: {:tls_alert, 'certificate expired'}}}

20 comments

#http_client #troubleshooting #https

0 8256 20

2020-06-03 14:36:38 UTC

Most Liked

kip

ex_cldr Core Team

According to curl the certificate has expired for at least https://api.amplitude.com/. There was an issue over the weekend with an expired root certificate from Comodo/Sectigo CA for https://openexchangerates.org as well. Perhaps these sites are using the same certificate chain. Have you checked their support site?

kip@Kips-iMac-Pro cldr_dates_times % curl "https://api.amplitude.com/"
curl: (60) SSL certificate problem: certificate has expired
More details here: https://curl.haxx.se/docs/sslcerts.html

curl failed to verify the legitimacy of the server and therefore could not
establish a secure connection to it. To learn more about this situation and
how to fix it, please visit the web page mentioned above.

Post #2

voltone

Exactly. The :ssl application treats the certificates sent by the server as a linear, unbreakable chain, ignoring any self-signed certificates that may have been sent because of misconfiguration. (Recent versions do allow for servers that send the chain out-of-order). Chain validation then tries to find a root certificate in the trust store that matches the issuer of the last certificate in that chain (furthest from the end-certificate).

This approach is problematic with cross-signed certificates, as this example shows. Most other implementation treat the intermediate CA certificates sent by the server as a pool from which to draw missing elements in the chain, and they try multiple paths until a valid one is found.

The most reliable way to resolve this is to reconfigure the server to not send the cross-signing intermediate CA once the issuing root CA is invalidated. If the CA trust store used on the Erlang/Elixir side includes the (self-signed) equivalent root CA, then verification of the shorter chain will succeed.

There is no easy way to fix this on the client side in a generic way. A workaround for specific cases should be possible by passing a ‘partial chain fun’.

Post #16

kip

ex_cldr Core Team

I ran the check on https://ssllabs.com which I think is well respected. It shows the intermediate and root certificates sent by the server as expired.

However, it also shows that there are three paths to establishing trust, one that includes the expired certificates and two that do not. As I understand it from some reading, the :ssl app is unable to resolve alternative trust paths and therefore will fail to validate the peer in more situations than other clients. (I know that is not precise, mostly because I only understand a very small amount of this topic).

It also suggests why you are seeing different outcomes from different clients.

Post #9

Last Post!

voltone

For Mint with CAStore, run mix deps.update castore: CAStore v0.1.6 removes the expired root CA, and the existing partial_chain hook in Mint does the rest.

Post #21