gmile

Certifi issue / regression in Rebar3?

I am on this host in CircleCI, debugging a failing Elixir code. The code fails to make an API request to Stripe. Particularly, the HTTP library fails to verify the certificate of the requested host. The HTTP client in use is hackney. The error I am getting is this one:

16:25:45.892 [info] TLS :client: In state :certify at ssl_handshake.erl:1950 generated CLIENT ALERT: Fatal - Unknown CA

In pursue to answer a question “well, what would be the CAs the system knows about”? I got down to this – a call to the following function returns an empty list:

:certifi.cacerts()
# => []

After studying the contents of the function on GitHub, I see that it reads the contents of a file containing certificates. That file exists on the file system, and indeed it contains certificates:

/home/circleci/project/_build/test/lib/certifi/priv/cacerts.pem

Running a somewhat equivalent Elixir code on this host will read the certificate bytes into memory just fine:

:public_key.pem_decode(File.read!(:certifi.cacertfile())) |> Enum.map(& elem(&1, 1)) |> Enum.reverse()

I am now puzzled as to why :certify would not return a list of certificates.

I find myself unable to dig deeper, as it appears that the contents of the function is changed during the compilation time using parse_transform, specifically using using ct_expand.

Digging into parse_transform, I see that there’s a way to get details about the kind of transform performed by enabling tracing:

A debugging facility exists: passing the option {ct_expand_trace, Flags} as an option,
or adding a compiler attribute -ct_expand_trace(Flags) will enable a form of call trace.

Flags' can be ’ (no trace) or [F]', where F’ is c' (call trace), r’ (return trace), or `x’ (exception trace)'.

Would someone please assist me in understanding how to enable the tracing exactly? Do I need to modify a file in :certifi somewhere (which I can totally afford to do), or pass a compiler option in command line to iex/elixir/elixirc somehow?

I am able to reliable reproduce this very issue by entering iex -S mix shell, and calling a dummy function Stripe.PaymentIntent.create(%{}).

Trivia:

elixir version: 1.11.4 (released several hours ago)
erlang version: 23.0 (erts-11.1)
hackney version: 1.17.0
certifi version: 2.5.3

8 comments

#erlang #troubleshooting #rebar3 #certificates

1 2203 8

2021-03-17 16:33:34 UTC

Marked As Solved

axelson

Scenic Core Team

Have you tried the latest version of Hackney and Certifi?

Certifi 2.6.1 was released today:

Erlang Certifi 2.6.1, providing SSL Mozilla CA bundles to your application has been released with updated certificates and compatibility with OTP >= 23.

And Hackney 1.17.2 was also released today:

use parse_trans 3.3.1 only (fix compatibility with Erlang < 21)
bump certifi version
Allow merging of SSL opts

From: hackney/NEWS.md at master · benoitc/hackney · GitHub

They seem potentially related. Also I would upgrade from Erlang 23.0 to 23.2 although that probably won’t help this problem.

Post #5

Also Liked

gmile

Running the following code in CircleCI containers for Elixir 1.11.3 and 1.11.4 depicts the issue:

mix local.rebar --force
wget -qO- https://github.com/certifi/erlang-certifi/archive/2.5.3.tar.gz | tar xvz
cd erlang-certifi-2.5.3
~/.mix/rebar3 compile
elixir -pa _build/default/lib/certifi/ebin -e "IO.inspect(:certifi.cacerts())"

Container with elixir 1.11.3 & rebar 3.13.1:

docker run -it cimg/elixir:1.11.3 sh -c 'mix local.rebar --force; wget -qO- https://github.com/certifi/erlang-certifi/archive/2.5.3.tar.gz | tar xvz; cd erlang-certifi-2.5.3; ~/.mix/rebar3 compile; elixir -pa _build/default/lib/certifi/ebin -e "IO.inspect(:certifi.cacerts())"'
# a list of binaries, suppressed for brevity

Container with elixir 1.11.4 & rebar 3.14.4:

docker run -it cimg/elixir:1.11.4 sh -c 'mix local.rebar --force; wget -qO- https://github.com/certifi/erlang-certifi/archive/2.5.3.tar.gz | tar xvz; cd erlang-certifi-2.5.3; ~/.mix/rebar3 compile; elixir -pa _build/default/lib/certifi/ebin -e "IO.inspect(:certifi.cacerts())"'
# => []

cimg/elixir is an image maintained by CircleCI, available here: cimg/elixir - Docker Image.

Post #1