Docker build: failed to compute cache key: "/app/_build/rel/my_app" not found: not found

Questions / Help
,
1

I’m trying to get a working Dockerfile – I started with mix phx.gen.release --docker and that was a great start. I had to tweak a few things (as expected). I’m figuring out how to get an SSH key in there (using Use Your local SSH Keys Inside a Docker Container | by David Barral | Trabe | Medium) so I can checkout private packages. I added a chunk something like this:

RUN useradd -m user
RUN mkdir -p /home/user/.ssh
COPY id_rsa /home/user/.ssh/id_rsa
RUN chown -R user:user /home/user/.ssh
RUN echo "Host *.github.com\n\tStrictHostKeyChecking no\n" >> /home/user/.ssh/config
RUN echo "github.com,140.82.113.3 ssh-rsa xxxxxxx\n" >> /home/user/.ssh/known_hosts
USER user

And that almost gets the thing to build via docker build .

 => ERROR [stage-1 6/6] COPY --from=builder --chown=nobody:root /app/_build/rel/my_app ./                                                    0.0s
------
 > [stage-1 6/6] COPY --from=builder --chown=nobody:root /app/_build/rel/my_app ./:
------
failed to compute cache key: "/app/_build/rel/my_app" not found: not found

I have a custom build location:

  defp releases do
    [
      my_app: [
        include_executables_for: [:unix],
        steps: [:assemble, :tar],
        overlays: ["envs/", "priv/", "config/"],
        path: "_build/rel"
      ]
    ]
  end

But I can’t quite figure out what needs to change to make this work. Can anyone point me in the right direction?

2

Ack, I just needed to update the last COPY:

COPY --from=builder --chown=nobody:root /app/_build/rel ./
3

FYI, I know you figured this out, but as an aside, you might consider using ssh-keyscan to populate that known_hosts file, rather than a single IP address. Of course this is at build-time, so the public key and/or addresses may change at any time after the image is built, so you may consider doing it at runtime, but I thought you might find it useful either way :slight_smile:

1 Like
4

Thanks, that’s a good tip. I’m still having trouble getting my private repos downloaded – it was working when I copied the entire known_hosts and everything else from .ssh/ into the container, but that’s not the Dockerfile I want to go with for many reasons.

My current block is:

RUN mkdir -p /root/.ssh
RUN echo "${SSH_KEY}\n" > /root/.ssh/id_rsa
RUN echo "Host github.com\n\tStrictHostKeyChecking no\n\tUser git\n\tIdentityFile /root/.ssh/id_rsa" >> /root/.ssh/config
RUN ssh-keyscan github.com > /root/.ssh/known_hosts
RUN chmod 0700 /root/.ssh
RUN chmod 600 /root/.ssh/id_rsa

that happens right before mix local.hex and mix deps.get but it’s choking when it tries to get a private repo:

#22 0.538 * Getting foo (git@github.com:xyz/foo.git - 4a501b76aee58ece9015410950a204a37f456fd8)
#22 0.835 Warning: Permanently added the ECDSA host key for IP address '140.82.112.3' to the list of known hosts.
#22 0.985 Load key "/root/.ssh/id_rsa": invalid format
#22 0.985 git@github.com: Permission denied (publickey).
#22 0.987 fatal: Could not read from remote repository.
#22 0.987
#22 0.987 Please make sure you have the correct access rights
#22 0.987 and the repository exists.
#22 0.990 ** (Mix) Command "git --git-dir=.git fetch --force --quiet --progress" failed

Any ideas? Is anyone maintaining a repo of Elixir/Phoenix Docker and docker-compose files?

5

I’ve actually dealt with exactly this problem in some Erlang projects! There’s a couple ways to do it

  1. Mount your private key in the container at build time. I don’t really like this, especially because the key may be called something else, and obviously you don’t want to leak it. There’s actually also RUN --mount=type=ssh which it seems you can use, but I haven’t used it, and it seems a bit trickier to use in CI (unlike the next option). However, it likely involves minimal cleanup, so it might be worth a whirl.
  2. The more complicated, but in my opinion better, way is to do something like this
    a. Generate a personal access token. I started doing this before they allowed repo-scoped tokens so I haven’t tried it, but they do exist these days. You can mount this into your dockerfile using RUN --mount=type=secret
    b. Tell git to always use https for dependencies: git config --global url."https://x-access-token:$(cat /run/secrets/github_pat)@github.com/".insteadOf "git@github.com:" (obviously replacing github_pat with whatever you call your secret). When mix goes to download dependencies from Github, this will force it to cut over and use your authorized route. In modern versions of git there is also http.extraheader as a config option, and I believe you can do Authorization: Basic <b64 of x-access-token:access-token> but this config option was added in a newer version of Git, so if you’re using an older one (e.g. in Debian or an older version of Ubuntu), you may not have it. I figured this out by reading the source code if this Github Action. GitHub - actions/checkout: Action for checking out a repo (seems to be documented a bit here, too checkout/0153-checkout-v2.md at 25a956c84d5dd820d28caab9f86b8d183aeeff3d · actions/checkout · GitHub)

With either of these options, I would either do this in a separate build stage (as it seems you’re already doing), or be very careful to clean up my git config and/or ssh key after doing so; you don’t want these tokens in the final built container.

6

This is making me crazy. Moreso because it’s so bloody difficult to see what Docker is doing.

I’ve opted to use deployment keys for the half-dozen private repos in use. I know that’s a lot of keys, but in theory it’s straight forward.

The first problem here is that Github seems to expect that you’ll use aliases to distinguish one “host” from the next. E.g.

$ git clone git@github.com-repo-1:OWNER/repo-1.git

instead of

$ git clone git@github.com:OWNER/repo-1.git

but none of their examples seem particularly relevant because in most apps you’re not gonna be issuing git commands directly: they’re gonna be issued on your behalf by the package manager. So the docs there are a step removed from the relevant use case I think most users have.

So, although it would be possible to modify the mix.exs to use aliases instead of the regular hostnames, e.g.

{:foo, git: "git@github.com-repo-1:OWNER/foo.git", tag: "v1.2.3"}

In reality, that’s not practical because it would mean that regular development flows could no longer run mix deps.get unless they edited their SSH config file to match the one needed by the Dockerfile for deployment.

So the only approach that seemed possible and wouldn’t prevent regular development operations was to list all the deployment keys in the SSH config, e.g.

# /root/.ssh/config
IdentityFile=/root/.ssh/deploy_key1
IdentityFile=/root/.ssh/deploy_key2
# ... etc...

This is admittedly inefficient because each attempt to authenticate against a private repo will involve a half-dozen failed attempts before it finds the right SSH key. But at least this should work.

I copy stuff over via:

RUN mkdir -p /root/.ssh
COPY docker/ssh_config /root/.ssh/config
RUN ssh-keyscan github.com > /root/.ssh/known_hosts
RUN chmod 0700 /root/.ssh
RUN chmod 600 /root/.ssh/*

However, I haven’t been able to make this work at all. I have tried many variations but the most I get is the following

#21 0.742 no such identity: /root/.ssh/foo_deployment_key: No such file or directory
#21 0.742 git@github.com: Permission denied (publickey).
#21 0.743 fatal: Could not read from remote repository.

if I rely on the contents of the .ssh/config file that I have copied into /root/.ssh/config,

or

Error connecting to agent: No such file or directory

if I try to manually add the keys via statements like:

RUN ssh-add /root/.ssh/foo_deployment_key