2-way-ssl (Mutual TLS) with HTTPoison and Hackney

quatermain · May 3, 2020, 5:58pm

Hello,

I have to do requests to some APIs with 2-way-ssl, so every request has to by signed with our cert and private key. We uploaded our certificate to 3rd party servers which provide these APIs.

We put this options into HTTPoison call:

[{:Certificate, cert_der, :not_encrypted}] =
      File.read!("/cert/cert.pem") |> :public_key.pem_decode()

    key = File.read!("/cert/key.pem") |> :public_key.pem_decode() |> hd |> Tuple.delete_at(2)

    [
      ssl: [
        versions: [:"tlsv1.2"],
        verify: :verify_peer,
        cacertfile: CAStore.file_path(),
        cert: cert_der,
        key: key,
        depth: 3,
        customize_hostname_check: [
          match_fun: :public_key.pkix_verify_hostname_match_fun(:https)
        ]
      ]
    ]

When we do request, it looks like server doesn’t know how to process it. Because their api should response error in json format and we got only this:

<html>\n<body>\n<h1>Fault</h1>env:Receiverfault:MessageBlocked</body>\n</html>\n

I tried make request with Postman (NodeJS) with same certificate and private key and it works without problem.

Any ideas where can be problem? I asked on slack and @voltone gave me some good advice but I’m a little bit lost and looks like I did something wrong.

Thanks for any help or suggeston

ityonemo · May 3, 2020, 6:06pm

Maybe try using the certfile and keyfile (instead of cert and key) options instead of decoding them yourself

ityonemo · May 3, 2020, 6:08pm

Also check this out: https://github.com/benoitc/hackney/issues/587

voltone · May 3, 2020, 6:15pm

I was thinking about the same root cause: missing intermediate certificates for the client cert. But in that case I would expect the ‘cert.pem’ file to contain more than one certificate, and that would cause the pattern match on the first line to fail.

@quatermain are these the exact files you used with Postman? Or is there another file with additional CA certificates to go with your client certificate?

See also my reply in this thread.

quatermain · May 3, 2020, 6:35pm

These files are same as in Postman

I tried, same response.

I will try this and others tomorrow. It’s too late now here. Thanks guys

voltone · May 3, 2020, 6:45pm

Hmm, so in that case it wouldn’t be an issue of missing intermediates.

The server’s response is not very helpful. If you make a request without client certificate at all, does the server throw a TLS alert, or do you get the same HTML error? I’m trying to understand whether the server is configured to handle mutual TLS auth issues at a higher protocol layer. Most (but not all) servers would reply with a TLS alert. Maybe everything is fine on the TLS layer, and you have a missing HTTP header (“Accept: application/json” ?).

There isn’t anything to try here if you do not have any intermediate CA certificates…

tangui · May 3, 2020, 9:08pm

Also you may try with curl, setting verbosity level to its maximum (-v) to determine:

if TLS client authentication is optional or disabled
if TLS client authentication actually happens
if not (which means TLS client authentication is optional), why it fails

Feel free to post it here, it’s usually quite informative. Beware before posting results here, I’m don’t remember whether the debug messages contain the private key or not.

quatermain · May 4, 2020, 12:50pm

You are the winner. Their API documentation and Sandbox API use for all api calls “application/x-www-form-urlencoded” and their Production API use “application/json” but only for auth api call. Others look like use www-form.

FYI - API owner is group of banks, about 7 countries and 7 banks in EU. So… I will write horror/comedy book after all my NDA will end.

quatermain · May 4, 2020, 3:00pm

Update: not done, I got json response with message ID and status code 400, no other information. I thought it’s working because I need id from this response. But I need consent id and not message id. But it looks like problem with their system. Not sure where can be problem when I got error with:

env:Receiverfault:MessageBlocked

or another error with message id.

So this pain is still there and I will have to resolve it ASAP . I contacted support last week so I hope they will get us something this week.

Thanks guys

quatermain · May 5, 2020, 5:49am

Update: support gave answer. Sandbox api use another API than production. We got new documentation for production API, so we have to update our code and test it on production.

quatermain · May 7, 2020, 11:42am

another update. We tried API of another Bank, where has to be same (hope so) Mutual TLS and we got:

[info] TLS client: In state cipher received SERVER ALERT: Fatal - Handshake Failure

and

 ** (OAuth2.Error) {:tls_alert, {:handshake_failure, 'received SERVER ALERT: Fatal - Handshake Failure'}}

with this setup:

[
      ssl: [
        versions: [:"tlsv1.2"],
        verify: :verify_peer,
        cacertfile: CAStore.file_path(), #"/app/lib/castore-0.1.5/priv/cacerts.pem"
        certfile: "/cert/cert.pem",
        keyfile: "/cert/key.pem",
        depth: 3,
        customize_hostname_check: [
          match_fun: :public_key.pkix_verify_hostname_match_fun(:https)
        ]
      ]
    ]

kip · May 8, 2020, 4:57am

Wouldn’t the parameters need to be charlists instead of binaries?

quatermain · May 15, 2020, 3:04pm

So we fixed it. We did this:

changed this:

:hackney.request(method, url, headers, body, req_opts)

to this:

HTTPoison.request(method, url, body, headers, req_opts)

This ssl setup with RSA private key:

[
      ssl: [
        versions: [:"tlsv1.2"],
        verify: :verify_peer,
        cacertfile: CAStore.file_path(),
        certfile: config(:certificate),
        keyfile: config(:private_key),
        ciphers: :ssl.cipher_suites(:all, :"tlsv1.2"),
        depth: 3,
        customize_hostname_check: [
          match_fun: :public_key.pkix_verify_hostname_match_fun(:https)
        ]
      ]
    ]

and the most important part in our mix file:

{:hackney, github: "benoitc/hackney", override: true}

Thanks everybody

kip · May 15, 2020, 3:08pm

I admire the perseverance, well done.

quatermain · May 17, 2020, 7:56am

we had to fix it. I didn’t want to change language and it’s for our new product for PSD2 application. Currently we are first in country with license and a lot of customers ask for it for last year. So we have big line of customers waiting for it. Big thanks for community which help us a lot.

ityonemo · May 17, 2020, 6:06pm

If now you have a proof of concept launched you might want to consider testing out refactoring with mint/mojito, when you have time. Good luck!

quatermain · May 18, 2020, 6:43am

I like Mint and Mojito looks very good. But we are not done yet, unfortunately. We have to struggle with custom implementation of Oauth2 for every bank and make it works. We need 10 banks for beginning. Some have same API, just changed endpoint and others have custom implementations. There are about 4 main standards (Berlin Group Standard, Poland Standard, Slovakia standard,…) which we have to implement to cover our regions. We hired new junior and I gave him test project where he has to implement Mint with OpenStreet Map API, so he has to learn messaging and so on. It’s very good for him to use Mint.

ityonemo · May 18, 2020, 6:54am

Perfect project for a junior!