toddholmberg

toddholmberg

Get SSL certificate with an HTTPoison or :hackney request

Hello. I am trying to find the best way forward doing the following:

  1. Make an HTTPS request to an arbitrary URL.
  2. Make this request using a proxy.
  3. Be able to set the TLS/SSL protocol used for this connection.
  4. Be able to set the protocol-appropriate cipher used for this connection.
  5. Return the associated SSL certificate.

HTTPoison allows setting SSL options via its hackney foundation, but does not have return the SSL certificate with the response.

The Erlang :ssl library will allow setting the protocol and cipher for a request, but I am having trouble navigating the specifics of how to do this request via a proxy.

Has anyone had any experience making such a request and getting the associated SSL certificate? Thanks in advance for any advice.

Marked As Solved

voltone

voltone

Most HTTP clients abstract away the connection layer: they let you pass in the options to tune the connection establishment, but when they return the response they don’t include any connection information. Even Mint, which give you quite a lot of control over the lower layers, doesn’t let you extract the server certificate using a public API.

Having said that, it is possible to kind of make this work with HTTPoison. First of all, we’ll use ‘async’ mode, in which the response is delivered to a process mailbox. Then we’ll hook into the certificate verification callback and deliver the server cert (if it is valid) to the same mailbox in a custom message:

HTTPoison.get(
  "https://blog.voltone.net/",
  [],
  stream_to: self(),
  proxy: {"localhost", 8080},
  hackney: [
    ssl_options: [
      versions: [:"tlsv1.2"],
      ciphers: :ssl.cipher_suites(:default, :"tlsv1.2"),
      cacertfile: :certifi.cacertfile(),
      depth: 3,
      verify_fun: {fn
        _, {:bad_cert, _} = reason, _ ->
          {:fail, reason}

        _,{:extension, _}, state ->
          {:unknown, state}

        _, :valid, state ->
          {:valid, state}

        peer_cert, :valid_peer, pid ->
          if :public_key.pkix_verify_hostname(peer_cert, [dns_id: 'blog.voltone.net'], match_fun: :public_key.pkix_verify_hostname_match_fun(:https)) do            
            send(pid, {:peer_cert, peer_cert})
            {:valid, pid}
          else
            {:fail, :hostname_check_failed}
          end
      end, self()}
    ]
  ]
)

Update the proxy address as needed, and remember to update the destination hostname in two places: the URI at the top, and the hostname verification function further down. Of course you could wrap this in a function that takes care of this automatically.

The result might look something like this:

iex(1)> HTTPoison.get(...)
{:ok, %HTTPoison.AsyncResponse{id: #Reference<0.1211898004.3575906308.200659>}}
iex(2)> flush
{:peer_cert,
 {:OTPCertificate,
  #...
 }
}
%HTTPoison.AsyncStatus{
  code: 200,
  id: #Reference<0.1211898004.3575906308.200659>
}
%HTTPoison.AsyncHeaders{
  headers: [
    #...
  ]
}
#...

The certificate is sent as an Erlang record. You can convert it to other formats (DER, PEM) using the :public_key module.

Finally, keep in mind that not every HTTP request necessarily results in a TLS handshake, and therefore a peer certificate message. The HTTP client may keep the connection open for reuse by multiple HTTP requests, and even if a new connection is established, TLS session resumption may cause the handshake to be skipped (pass reuse_sessions: false in the ssl_options to prevent this).

Also Liked

voltone

voltone

Well, if you don’t need the HTTP response at all you can take a few shortcuts. Drop the stream_to: self() option, ignore the response from HTTPoison.get (or check only for connection errors) and then see if you got a certificate:

receive do
  {:peer_cert, peer_cert} ->
    {:ok, peer_cert}
after
  0 ->
    {:error, "No TLS handshake, or handshake failed"}
end

You can also change the method to HEAD, to avoid transferring data you’re going to throw away anyway.

At this point I’m wondering whether you need an HTTP client at all. You could just open a TCP connection to the proxy, send "CONNECT some.host.name:443 HTTP/1.1\r\n\r\n", receive the "HTTP/1.1 200 Connection established\r\n\r\n" reply, then upgrade the connection to TLS with :ssl.connect/2 with all the necessary ssl_options. If all goes well, call :ssl.peercert/1 to get the peer certificate.

{:ok, tcp} = :gen_tcp.connect('proxy.host.name', 8080, active: false)
:gen_tcp.send(tcp, "CONNECT some.host.name:443 HTTP/1.1\r\n\r\n")
{:ok, "HTTP/1.1 200" <> _} = :gen_tcp.recv(tcp, 0)
{:ok, ssl} = :ssl.connect(tcp, [
  versions: [:"tlsv1.2"],
  ciphers: :ssl.cipher_suites(:default, :"tlsv1.2"),
  server_name_indication: 'some.host.name',
  verify: :verify_peer,
  cacertfile: :certifi.cacertfile(),
  depth: 3,
  customize_hostname_check: [
    match_fun: :public_key.pkix_verify_hostname_match_fun(:https)
  ]
])
{:ok, der} = :ssl.peercert(ssl)

You should now have the peer certificate in DER binary format.

toddholmberg

toddholmberg

Thank you, @voltone! This is exactly the help I needed. I will also be using your excellent X509 package to process the resulting certificate.

toddholmberg

toddholmberg

One more boost for @voltone : couple this answer with the recursive approach in this blog post series by @alvises and you can pattern match on just the certificate and ignore the rest of the response chunks.

Last Post!

toddholmberg

toddholmberg

Thanks, again, @voltone.. I really appreciate your help with this.

Where Next?

Popular in Questions Top

minhajuddin
I have seen a lot of code which picks the first element from a list using Enum.at(0) instead of List.first. Is there a reason why people ...
New
lanycrost
Hi everyone! I need implement if…else if…else condition from my elixir code, and anymore of this control flow structures not work proper...
New
joeerl
Hello again - after a longish gap I’ve decided I really must dig into Elixir and see what’s been happening here - so I have a few questio...
New
greenz1
I have a phoenix application from which a user can download multiple(5-6) files of size 1MB. I couldn’t find anything related to sending ...
New
sen
Hi All, I set a environment variables in dev.exs , like below code. when i start server, how can i set the ${enable} value? thanks. d...
New
stefanluptak
Hello everybody, usually, I use a 29" ultra-wide monitor for VSCode which can easily accomodate explorer (files panel) + file with code ...
New
freewebwithme
Using vs code and installed ElixirLS: support and debugger. And I got an error popped up on start up says Failed to run ‘elixir’ comma...
New

Other popular topics Top

rms.mrcs
Hi, I need to transform a list of numbers into a map where the keys are the indexes and the values are the original values of the list. ...
New
KronicDeth
Elixir plugin for JetBrain’s IntelliJ Platform (including Rubymine) This is a plugin that adds support for Elixir to JetBrains IntelliJ...
289 36820 110
New
Darmani72
If I have a post route which an argument: post /my_post_route/:my_param1, MyController.my_post_handler How would get the post params ...
New
chrismccord
Phoenix 1.4.0 released Phoenix 1.4 is out! This release ships with exciting new features, most notably with HTTP2 support, improved deve...
688 31586 112
New
sergio_101
I am VERY much an elixir newbie. I have taken one elixir course and one phoenix course on Udemy. During that course, I saw the instructor...
New
sergio
Kind of like when jquery came out, it was super necessary. Existing drag and drop libraries have a bunch of baggage to support old browse...
New

We're in Beta

About us Mission Statement