Hackney can't find :certifi modules after code change, need to run deps.compile

Hello,

I’m new to elixir and I’ve been writing a crawler using GenStage, where there is Queue and Fetcher, the fetcher uses :httpoison (1.4.0).

When I run it after doing some code changes, I get the error from hackney that it can’t find certifi module.
I have to run mix deps.compile and when I run it again, it works until another change code.

Is it expected behaviour or is it some sort of a bug?

Here is the error and deps and versions

mix run scripts/crawler_cli.exs https://www.smartwalls.cz
Crawlin domain https://www.smartwalls.cz

10:49:51.797 [error] GenServer #PID<0.209.0> terminating
** (UndefinedFunctionError) function :certifi.cacerts/0 is undefined (module :certifi is not available)
    (certifi) :certifi.cacerts()
    (hackney) /Users/dal/Work/Analyticoo/crawler/deps/hackney/src/hackney_connect.erl:326: :hackney_connect.ssl_opts_1/2
    (hackney) /Users/dal/Work/Analyticoo/crawler/deps/hackney/src/hackney_connect.erl:273: :hackney_connect.do_connect/5
    (hackney) /Users/dal/Work/Analyticoo/crawler/deps/hackney/src/hackney_connect.erl:45: :hackney_connect.connect/5
    (hackney) /Users/dal/Work/Analyticoo/crawler/deps/hackney/src/hackney.erl:329: :hackney.request/5
    (httpoison) lib/httpoison/base.ex:746: HTTPoison.Base.request/6
    (crawler) lib/crawler/fetcher.ex:38: Crawler.Fetcher.fetch/1
    (crawler) lib/crawler/fetcher.ex:22: anonymous fn/3 in Crawler.Fetcher.handle_events/3
Last message: {:"$gen_consumer", {#PID<0.208.0>, #Reference<0.779772765.3343122435.147681>}, ["https://www.smartwalls.cz/"]}
State: {#PID<0.207.0>}
^C
BREAK: (a)bort (c)ontinue (p)roc info (i)nfo (l)oaded
       (v)ersion (k)ill (D)b-tables (d)istribution

 $ mix deps.compile
===> Compiling parse_trans
===> Compiling mimerl
===> Compiling metrics
===> Compiling unicode_util_compat
===> Compiling idna
===> Compiling certifi
===> Compiling hackney
===> Compiling mochiweb
src/mochiweb_socket.erl:91: Warning: ssl:ssl_accept/2: deprecated; use ssl:handshake/2 instead

$ mix run scripts/crawler_cli.exs https://www.smartwalls.cz
Crawlin domain https://www.smartwalls.cz
1 of 2 (50.0%)
1 of 2 (50.0%)
1 of 2 (50.0%)
1 of 2 (50.0%)
1 of 2 (50.0%)
9 of 59 (15.254237288135593%)
22 of 59 (37.28813559322034%)
33 of 59 (55.932203389830505%)
36 of 59 (61.016949152542374%)
37 of 59 (62.71186440677966%)
41 of 59 (69.49152542372882%)
43 of 84 (51.19047619047619%)
47 of 85 (55.294117647058826%)
47 of 85 (55.294117647058826%)
49 of 85 (57.647058823529406%)

Erlang/OTP 21 [erts-10.0.5] [source] [64-bit] [smp:4:4] [ds:4:4:10] [async-threads:1] [hipe]
Elixir 1.7.2 (compiled with Erlang/OTP 19)

* gen_stage 0.14.1 (Hex package) (mix)
  locked at 0.14.1 (gen_stage) 9d46723f
 * certifi 2.4.2 (Hex package) (rebar3)
  locked at 2.4.2 (certifi) 75424ff0
* hackney 1.14.3 (Hex package) (rebar3)
  locked at 1.14.3 (hackney) b5f6f5dc
* httpoison 1.4.0 (Hex package) (mix)
  locked at 1.4.0 (httpoison) e0b3c2ad

Thanks in advance for any help.

1 Like

So this seems to be already reported issue, currently being investigated … https://github.com/benoitc/hackney/issues/528

1 Like