n1c0

n1c0

Hi Guys,

I am trying to do a POST request via hackney (1.14.3).

The curl query is as follow:

curl -v \
--cert /Users/nico/projects/pfx/cert.pem \
--key /Users/nico/projects/pfx/key.pem \
--pass PASSWORD \
--insecure \
--header "Content-type: application/soap+xml; charset=utf-8; action=\"http://www.portalfiscal.inf.br/nfe/wsdl/NFeDistribuicaoDFe\"" \
--header "Accept: application/soap+xml; charset=utf-8" \
--data "<?xml version=\"1.0\" encoding=\"UTF-8\"?>...</soap12:Envelope>" \
https://www1.nfe.fazenda.gov.br/NFeDistribuicaoDFe/NFeDistribuicaoDFe.asmx

which gives the following results:

* Trying 200.198.239.181...
* TCP_NODELAY set
* Connected to www1.nfe.fazenda.gov.br (200.198.239.181) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* Cipher selection: ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
* successfully set certificate verify locations:
* CAfile: /etc/ssl/cert.pem
CApath: none
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Client hello (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS change cipher, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES256-SHA384
* ALPN, server did not agree to a protocol
* Server certificate:
* subject: C=BR; O=ICP-Brasil; OU=Secretaria da Receita Federal do Brasil - RFB; OU=ARSERPRO; OU=RFB e-Servidor A1; CN=www1.nfe.fazenda.gov.br
* start date: Sep 19 16:49:12 2018 GMT
* expire date: Sep 19 16:49:12 2019 GMT
* issuer: C=BR; O=ICP-Brasil; OU=Secretaria da Receita Federal do Brasil - RFB; CN=Autoridade Certificadora do SERPRORFB SSL
* SSL certificate verify result: unable to get local issuer certificate (20), continuing anyway.
> POST /NFeDistribuicaoDFe/NFeDistribuicaoDFe.asmx HTTP/1.1
> Host: www1.nfe.fazenda.gov.br
> User-Agent: curl/7.54.0
> Content-type: application/soap+xml; charset=utf-8; action="http://www.portalfiscal.inf.br/nfe/wsdl/NFeDistribuicaoDFe"
> Accept: application/soap+xml; charset=utf-8
> Content-Length: 790
> 
* upload completely sent off: 790 out of 790 bytes
* TLSv1.2 (IN), TLS handshake, Hello request (0):
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Request CERT (13):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Certificate (11):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS handshake, CERT verify (15):
* TLSv1.2 (OUT), TLS change cipher, Client hello (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS change cipher, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Finished (20):
< HTTP/1.1 200 OK
< Cache-Control: private, max-age=0
< Content-Type: application/soap+xml; charset=utf-8
< Server: Microsoft-IIS/8.5
< X-AspNet-Version: 4.0.30319
< X-Powered-By: ASP.NET
< Date: Thu, 13 Dec 2018 21:17:00 GMT
< Content-Length: 6380

now when i am trying to do the same with hackney in elixir (1.7.4), i am getting an {:error, :closed}:

url = "https://www1.nfe.fazenda.gov.br/NFeDistribuicaoDFe/NFeDistribuicaoDFe.asmx"
xml_soap = "<?xml version=\"1.0\" encoding=\"UTF-8\"?>...</soap12:Envelope>"

headers = 
  [
    {"Content-type", "application/soap+xml; charset=utf-8; action=\"http://www.portalfiscal.inf.br/nfe/wsdl/NFeDistribuicaoDFe\""},
    {"Accept", "application/soap+xml; charset=utf-8"}
  ]

opts = [
  :insecure,
  recv_timeout: 30_000,
  ssl_options: [
    certfile: "/Users/nico/projects/pfx/cert.pem",
    keyfile: "/Users/nico/projects/pfx/key.pem",
    password: 'PASSWORD',
    versions: [:'tlsv1.2']
  ]
]

leading to:

iex(8)> :hackney.request(:post, url, headers, xml_soap, opts)
{:error, :closed}

I guess the issue must be in the SSL negotiation, but don’t know how to debug this…any idea?

thanks!

Showing Posts 1 to 10

chensan

chensan

You might need to start ssl yourself, try:

:ssl.start
:hackney.request(:post, url, headers, xml_soap, opts)
voltone

voltone

I don’t have a valid client certificate, so I can’t fully replicate your scenario, but when I connect to that address with Hackney I see the same response. The important thing to note is that the connection gets closed after the TLS handshake completes.

Does the ‘curl’ command actually show a response from the server? The output you quoted only shows curl trying to send a request.

voltone

voltone

Also, as far as I can tell the server does not request a client certificate at all…

n1c0

n1c0 OP

Hi,

if i do the curl command without the --key & --cert params, I get the following HTTP/1.1 403 Forbidden response:

*   Trying 200.198.239.181...
* TCP_NODELAY set
* Connected to www1.nfe.fazenda.gov.br (200.198.239.181) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* Cipher selection: ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/cert.pem
  CApath: none
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Client hello (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS change cipher, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES256-SHA384
* ALPN, server did not agree to a protocol
* Server certificate:
*  subject: C=BR; O=ICP-Brasil; OU=Secretaria da Receita Federal do Brasil - RFB; OU=ARSERPRO; OU=RFB e-Servidor A1; CN=www1.nfe.fazenda.gov.br
*  start date: Sep 19 16:49:12 2018 GMT
*  expire date: Sep 19 16:49:12 2019 GMT
*  issuer: C=BR; O=ICP-Brasil; OU=Secretaria da Receita Federal do Brasil - RFB; CN=Autoridade Certificadora do SERPRORFB SSL
*  SSL certificate verify result: unable to get local issuer certificate (20), continuing anyway.
> POST /NFeDistribuicaoDFe/NFeDistribuicaoDFe.asmx HTTP/1.1
> Host: www1.nfe.fazenda.gov.br
> User-Agent: curl/7.54.0
> Content-type: application/soap+xml; charset=utf-8; action="http://www.portalfiscal.inf.br/nfe/wsdl/NFeDistribuicaoDFe"
> Accept: application/soap+xml; charset=utf-8
> Content-Length: 790
> 
* upload completely sent off: 790 out of 790 bytes
* TLSv1.2 (IN), TLS handshake, Hello request (0):
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Request CERT (13):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Certificate (11):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Client hello (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS change cipher, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Finished (20):
< HTTP/1.1 403 Forbidden
< Content-Type: text/html
< Server: Microsoft-IIS/8.5
< X-Powered-By: ASP.NET
< Date: Fri, 14 Dec 2018 09:31:06 GMT
< Content-Length: 1233
< 
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1"/>
<title>403 - Forbidden: Access is denied.</title>
<style type="text/css">
<!--
body{margin:0;font-size:.7em;font-family:Verdana, Arial, Helvetica, sans-serif;background:#EEEEEE;}
fieldset{padding:0 15px 10px 15px;} 
h1{font-size:2.4em;margin:0;color:#FFF;}
h2{font-size:1.7em;margin:0;color:#CC0000;} 
h3{font-size:1.2em;margin:10px 0 0 0;color:#000000;} 
#header{width:96%;margin:0 0 0 0;padding:6px 2% 6px 2%;font-family:"trebuchet MS", Verdana, sans-serif;color:#FFF;
background-color:#555555;}
#content{margin:0 0 0 2%;position:relative;}
.content-container{background:#FFF;width:96%;margin-top:8px;padding:10px;position:relative;}
-->
</style>
</head>
<body>
<div id="header"><h1>Server Error</h1></div>
<div id="content">
 <div class="content-container"><fieldset>
  <h2>403 - Forbidden: Access is denied.</h2>
  <h3>You do not have permission to view this directory or page using the credentials that you supplied.</h3>
 </fieldset></div>
</div>
</body>
</html>
* Connection #0 to host www1.nfe.fazenda.gov.br left intact

and now with the certfile/keyfile:

*   Trying 200.198.239.181...
* TCP_NODELAY set
* Connected to www1.nfe.fazenda.gov.br (200.198.239.181) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* Cipher selection: ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/cert.pem
  CApath: none
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Client hello (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS change cipher, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES256-SHA384
* ALPN, server did not agree to a protocol
* Server certificate:
*  subject: C=BR; O=ICP-Brasil; OU=Secretaria da Receita Federal do Brasil - RFB; OU=ARSERPRO; OU=RFB e-Servidor A1; CN=www1.nfe.fazenda.gov.br
*  start date: Sep 19 16:49:12 2018 GMT
*  expire date: Sep 19 16:49:12 2019 GMT
*  issuer: C=BR; O=ICP-Brasil; OU=Secretaria da Receita Federal do Brasil - RFB; CN=Autoridade Certificadora do SERPRORFB SSL
*  SSL certificate verify result: unable to get local issuer certificate (20), continuing anyway.
> POST /NFeDistribuicaoDFe/NFeDistribuicaoDFe.asmx HTTP/1.1
> Host: www1.nfe.fazenda.gov.br
> User-Agent: curl/7.54.0
> Content-type: application/soap+xml; charset=utf-8; action="http://www.portalfiscal.inf.br/nfe/wsdl/NFeDistribuicaoDFe"
> Accept: application/soap+xml; charset=utf-8
> Content-Length: 790
> 
* upload completely sent off: 790 out of 790 bytes
* TLSv1.2 (IN), TLS handshake, Hello request (0):
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Request CERT (13):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Certificate (11):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS handshake, CERT verify (15):
* TLSv1.2 (OUT), TLS change cipher, Client hello (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS change cipher, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Finished (20):
< HTTP/1.1 200 OK
< Cache-Control: private, max-age=0
< Content-Type: application/soap+xml; charset=utf-8
< Server: Microsoft-IIS/8.5
< X-AspNet-Version: 4.0.30319
< X-Powered-By: ASP.NET
< Date: Fri, 14 Dec 2018 09:35:08 GMT
< Content-Length: 6380
< 
<?xml version="1.0" encoding="utf-8"?><soap:Envelope xmlns:soap="http://www.w3.org/2003/05/soap-envelope" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema"><soap:Body><nfeDistDFeInteresseResponse xmlns="http://www.portalfiscal.inf.br/nfe/wsdl/NFeDistribuicaoDFe"><nfeDistDFeInteresseResult><retDistDFeInt xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" versao="1.01" xmlns="http://www.portalfiscal.inf.br/nfe"><tpAmb>1</tpAmb><verAplic>1.2.0</verAplic><cStat>138</cStat><xMotivo>Documento localizado</xMotivo><dhResp>2018-12-14T07:35:08-02:00</dhResp>...</soap:Body></soap:Envelope>

i’ve truncated a bit of the response body but i get what i need with curl

voltone

voltone

Ah, sorry, I just noticed the scrollbar alongside your original curl output, so it wasn’t truncated after all.

The issue is this: in the initial handshake the server does not send a client certificate request, and so the client does not send one (both curl and Hackney). Once the server has received the HTTP request it presumably decides that this particular request requires a certificate-based authentication and it attempts to renegotiate the session. In this second handshake it does request a client certificate:

If the renegotiation succeeds and the client presents a valid certificate, the server sends its response.

With Hackney, the renegotiation fails and the connection is closed.

This is an interesting scenario, I will look into it some more and report back…

voltone

voltone

Here’s what I’ve found so far:

The issue is not Hackney specific: the same thing happens when connecting with Erlang’s ssl API directly.

The issue is not related to the client certificate: the renegotiation only gets as far as the client sending the ClientHello, and the server immediately closes the connection. The server never gets to the point where it asks for the client certificate.

This appears to be a regression in renegotiation handling in recent OTP versions: when I tried the connection on OTP 21.0.9 the renegotiation succeeded (I don’t have a client certificate, but at least I do get the 403 response). On 21.1 and 21.2 it does not work.

n1c0

n1c0 OP

alright thanks for the feedback! I have downgraded to OTP 21.0.9 and run the command again, now getting the 200 response code & the content like in curl:

{:ok, 200,
 [
   {"Cache-Control", "private, max-age=0"},
   {"Content-Type", "application/soap+xml; charset=utf-8"},
   {"Server", "Microsoft-IIS/8.5"},
   {"X-AspNet-Version", "4.0.30319"},
   {"X-Powered-By", "ASP.NET"},
   {"Date", "Sat, 15 Dec 2018 16:27:39 GMT"},
   {"Content-Length", "6380"}
 ], #Reference<0.3983044557.4188012545.22511>}

Do you mind raising an issue for OTP 21.2 ? you must have more details than myself to explain what’s going on ?

Thanks!

voltone

voltone

Another update: I was able to reproduce the issue against OpenSSL. It turns out it only happens if the client sends data prior to the server’s renegotiation request. The test cases included in OTP for renegotiation between an Erlang client and OpenSSL server perform renegotiation prior to any data exchange.

Based on all the information gathered I opened an issue in the Erlang bug tracker:

If anyone wants to reproduce the issue, first launch an OpenSSL test server in one terminal window (you need a certificate and key, for example a self-signed certificate):

$ openssl s_server -cert server.pem -key server_key.pem -msg
Using default temp DH parameters
ACCEPT

In another terminal window, start an Erlang or Elixir shell and connect to the test server and send some data:

Erlang/OTP 21 [erts-10.1] [source] [64-bit] [smp:4:4] [ds:4:4:10] [async-threads:1] [hipe]

Interactive Elixir (1.7.3) - press Ctrl+C to exit (type h() ENTER for help)
iex(1)> :ssl.start
:ok
iex(2)> {:ok, s} = :ssl.connect('localhost', 4433, [])
{:ok,
 {:sslsocket, {:gen_tcp, #Port<0.7>, :tls_connection, :undefined},
  [#PID<0.128.0>, #PID<0.127.0>]}}
iex(3)> :ssl.send(s, "Hello!")   
:ok
iex(4)> 

At this point, type ‘R’ and hit Enter in the OpenSSL window.

voltone

voltone

Fix is now available as part of patch package 21.2.1:

n1c0

n1c0 OP

Perfect! thanks for your help on that matter

— All posts loaded —

Where Next? Top

Trending in Questions Top

Blokh
Hey guys, I’ve got a huge CSV ( around 10 GB ) that needs to be processed hourly Do you guys have any suggestions what is the best prac...
New
kszambelanczyk
Hello! Could someone please give me a help/sample code, how to delete a file from s3 using waffle/waffle_ecto from Phoenix app. I creat...
New
Onor.io
I have what I’ve heard referred to as a “lookup table” in my database. This is a way of assigning codes to common values. One common lo...
New
jaybe78
Hello, I’m developing a online persistent chat system (what’s app) like using elixir/dynamodb/aws for a mobile app(flutter). The diffic...
New
Trolleger
What approach to take when sending live updates to “random” users Hi! I have a question, I have a little chat app, and when I create a DM...
New
matt-savvy
Anyone here using Honeybadger? My Honeybadger account is being overwhelmed with noise from some bots. Seeing a lot of Bandit.HTTPError...
New
RemyXRenard
I’m seeing that a list inside a Kino.DataTable will be interpreted as a charlist, even if the Kino.configure() is set to charlists: :as_l...
New

Other Trending Topics Top

garrison
Hobbes is a low-level distributed database for the Elixir programming language. Hobbes provides a simple, safe, and scalable storage lay...
New
mcass19
ExRatatui lets you cook up rich terminal UIs in Elixir, powered by Rust’s ratatui via Rustler NIFs. Build interactive terminal applicatio...
New
Damirados
Hello everyone. After busy few months I am happy to announce v0.1.0 of Emerge &amp; Solve. They are GUI (Emerge) and State management (S...
New
netoum
Corex is an accessible, unstyled UI component library for Phoenix that integrates Zag.js state machines using Vanilla JavaScript and Live...
New
wintermeyer
There are three potential reasons for members of this forum to have a look at https://vutuv.de You are tired or annoyed of LinkedIn. Yo...
New
webofbits
Aludel - LLM Evaluation Workbench Aludel is an embeddable Phoenix LiveView dashboard for evaluating and comparing LLM prompts across mult...
New

We're in Beta

About us Mission Statement

Options

Thread Display Mode




Thread Preview

Skip Thread Previews