Broadway Kafka - (Mix) Could not compile dependency :snappyer, "escript.exe

Questions / Help
,
1

Hi Everyone,

I’m trying to use the Broadway Kafka producer but I’m facing compilation error on a transitive dependency called snappyer which is used by brod :

$ mix compile
===> Linking d:/workspace/Veritus/veritus_umbrella/_build/dev/lib/snappyer/priv/snappyer.dll
** (Mix) Could not compile dependency :snappyer, “escript.exe “c:/Users/didus/.mix/rebar3” bare compile –
paths d:/workspace/Veritus/veritus_umbrella/_build/dev/lib/*/ebin” command failed. Errors may have been
logged
above. You can recompile this dependency with “mix deps.compile snappyer”, update it with “mix
deps.update snappyer” or clean it with “mix deps.clean snappyer”

I had different errors before installing Visual Studio Build Tools beacause snippyer couldn’t compile its C++ files (even though I’m not supposed to have it installed on my PC as I’m using Elixir and not C++ !).

I installed last version of Elixir 1.13.2 but it didn’t change anything. I tried to rebar directely inside the deps/snappyer and got :

C:/Users/didus/.mix/rebar3 compile -v
←[0;32m===> Verifying dependencies…
←[0m←[0;32m===> Linking d:/workspace/Veritus/veritus_umbrella/deps/snappyer/priv/snappyer.dll
←[0m

when I look at the priv I can’t see any DLL inside. Am I missing something ? thanks a lot for your help

2

I haven’t worked on Windows for years but you need proper PATH environment variable configuration so the VS Build Tools are visible system-wide.

I don’t disagree with you but some libraries have native dependencies because the maintainers chose to have them. You can opt out by not installing the library. :man_shrugging:

I wish you luck because most maintainers only pay attention to UNIX-like systems for building native libraries (so macOS and Linux). But I’ve had success with compiling C/C++ deps on Windows in the past. Wasn’t exactly trivial but wasn’t very hard either.

3

thanks for your reply. The code gets compiled successfully but it seems it cannot generate the DLL needed by snappyer. Is there a way to exlude this dependency from broadway_kafka as I’m not planning tu use snappy for compression ?

4

for those who are getting the same error as mine you can visit this link : link

my C++ was targeting x86 compiler version whereas the snappyer rebar was looking for the x64 compiler version. Thanks a lot.

1 Like
5

@didus what did you do to solve the issue. I took a look at the link you suggested but I cannot find an answer. I already tried many things like running brew doctor, brew cleanup, brew upgrade, running some asdf commands to get other Erlang/Elixir versions and nothing.

In my case I was unable to compile my project and I got stuck in the next message:

#include <algorithm>
         ^~~~~~~~~~~
1 error generated.

** (Mix) Could not compile dependency :snappyer,...

I’m running my code un a Mac.
I’m using elixir 1.12 / erlang 24.3.4

I suspect is a Kafka problem because that error started when somebody in my work added that dependency.

6

Hi @catra,

snappyer is a transitive library used by broadway_kafka. It uses native code written in C++. In my case as I’m using Windows I couldn’t compile it because I had no C++ compiler on my machine. I struggled to have it to work by installing the MS C++ compiler but it didn’t. As I’m using visual code on windows I found an extention : Remote - WSL so that, the visual code is still on windows but the code is compiled and executed under Linux and this worked as linux has C++ compiler (GCC). I have tried on my Mac and it worked too.

Maybe you should check if you have gcc installed on your Mac ?