(Mix) Could not compile dependency :cowboy, "/root/.mix/rebar3 bare compile

I have a phoenix project hosted on aws and I’m using AWS codebuild to build the mix release.

I am able to run:

mix do deps.get, compile

successfully on my aws codebuild build.

But now I’m looking to cache my _build directory using:

cache:
  paths:
    - _build
    - deps
    - priv/plts

in my codebuild buildspec.yml and I get the following error:

...

===> Compiling cowboy
--
66 | src/cowboy_clear.erl:16: Warning: behaviour ranch_protocol undefined
67 |  
68 | ===> Compiling src/cowboy_http.erl failed
69 | src/cowboy_http.erl:154: can't find include lib "cowlib/include/cow_inline.hrl"; Make sure cowlib is in your app file's 'applications' list
70 | src/cowboy_http.erl:155: can't find include lib "cowlib/include/cow_parse.hrl"; Make sure cowlib is in your app file's 'applications' list
71 | src/cowboy_http.erl:482: undefined macro 'IS_TOKEN/1'
72 | src/cowboy_http.erl:628: undefined macro 'IS_WS/1'
73 |  
74 | src/cowboy_http.erl:462: function parse_method/4 undefined
75 | src/cowboy_http.erl:616: function parse_hd_name/4 undefined
76 |  
77 | src/cowboy_http.erl:487: Warning: function parse_uri/3 is unused
78 | src/cowboy_http.erl:504: Warning: function parse_uri_authority/3 is unused
79 | src/cowboy_http.erl:508: Warning: function parse_uri_authority/5 is unused
80 | src/cowboy_http.erl:533: Warning: function parse_uri_path/5 is unused
81 | src/cowboy_http.erl:543: Warning: function parse_uri_query/6 is unused
82 | src/cowboy_http.erl:552: Warning: function skip_uri_fragment/6 is unused
83 |  
84 | ==> samhstn
85 | ** (Mix) Could not compile dependency :cowboy, "/root/.mix/rebar3 bare compile --paths /codebuild/output/src496784613/src/github.com/samhstn/samhstn/_build/test/lib/*/ebin" command failed. You can recompile this dependency with "mix deps.compile cowboy", update it with "mix deps.update cowboy" or clean it with "mix deps.clean cowboy"
86 |  
87 | [Container] 2020/11/17 13:35:49 Command did not exit successfully MIX_ENV=test mix compile exit status 1

I have tried adding: {:cowlib, "~> 2.8.0"} to my deps (following this: https://github.com/phoenixframework/phoenix/issues/3903#issuecomment-646864037), but still have the same problem.

I think the problem may be that codebuild symlinks it’s cache to the target location and this causes cowboy to error on compile (see similar issue here: https://github.com/aws-samples/aws-codebuild-samples/issues/6#issue-414920460).

Running ls -al and ls -alL for _build and deps before mix compile gives the output:

# ls -al deps
lrwxrwxrwx 1 root root 101 Nov 18 16:35 deps -> /codebuild/local-cache/custom/6e2710a42bf47/deps

# ls -al _build
lrwxrwxrwx 1 root root 99 Nov 18 16:35 _build -> /codebuild/local-cache/custom/6e2710a42bf47/_build

# ls -alL _build
drwxr-xr-x 2 root root 4096 Nov 18 16:46 .
drwxr-xr-x 3 root root 4096 Nov 18 16:46 ..

# ls -alL deps
drwxr-xr-x 15 root root 4096 Nov 18 16:46 .
drwxr-xr-x  3 root root 4096 Nov 18 16:46 ..
drwxr-xr-x  3 root root 4096 Nov 18 16:46 cowboy
drwxr-xr-x  3 root root 4096 Nov 18 16:46 cowboy_telemetry
drwxr-xr-x  4 root root 4096 Nov 18 16:46 cowlib
drwxr-xr-x  3 root root 4096 Nov 18 16:46 jason
drwxr-xr-x  4 root root 4096 Nov 18 16:46 mime
drwxr-xr-x  5 root root 4096 Nov 18 16:46 phoenix
drwxr-xr-x  4 root root 4096 Nov 18 16:46 phoenix_html
drwxr-xr-x  4 root root 4096 Nov 18 16:46 phoenix_pubsub
drwxr-xr-x  4 root root 4096 Nov 18 16:46 plug
drwxr-xr-x  3 root root 4096 Nov 18 16:46 plug_cowboy
drwxr-xr-x  3 root root 4096 Nov 18 16:46 plug_crypto
drwxr-xr-x  3 root root 4096 Nov 18 16:46 ranch
drwxr-xr-x  3 root root 4096 Nov 18 16:46 telemetry

What is a potential way around this problem and what should I look to do to make caching between builds possible?

1 Like