Smallest docker container with elixir app

After docker released native support for multi-stage builds, I decided to make my own small docker container with app.

Below container is based on the one from @bitwalker. It is the smallest one I could come up with in one sitting, given quite limited knowledge of docker & distillery releases:

# Dockerfile
FROM bitwalker/alpine-elixir:1.4.4 as builder

ADD . /app

WORKDIR /app

ENV MIX_ENV=prod

RUN mix do deps.get, deps.compile, release

FROM alpine:3.6

RUN apk add --no-cache \
      ca-certificates \
      openssl-dev \
      ncurses-dev \
      unixodbc-dev \
      zlib-dev

WORKDIR /app

COPY --from=builder /app/_build/prod/rel/ritm/releases/0.0.1/ritm.tar.gz /app

ENV PORT=4000

RUN tar -xzf ritm.tar.gz; rm ritm.tar.gz

CMD ["bin/ritm", "foreground"]

It’s around 64Mb when pushed to docker hub. That’s a lot, although I don’t expect this to grow significantly (more app code & deps shouldn’t contribute that much).

I am aware guys from @Nerves-Core-Team build super-small linux images with the app installed, but I am not sure what could be derived from their work in order to build my own small container with docker.

How can I reduce the size of container even further?

2 Likes

@gmile You don’t need the -dev libs on the release image.

This should suffice:

 add --no-cache \
      ncurses-libs \
      zlib \
      openssl \
      ca-certificates

@wmnnd this dropped the size to 56 MB, thanks!

How big is the actual release tar file you copy onto the image? The
reason I am asking is that I built alpine from scratch with our erlang
release and it was 18MB where 2/3 (if I remember correctly) was our release.

You should be able to shrink this significantly.

How do you suggest shrinking the release size?

Normally you don’t. You can exclude debug information which will make it
much smaller (half the size?). Without debug information you cannot
do things like tracing and other useful introspection in production. But
if you need the smallest available this is an option.

Generally the release size is the size your image will be + 5MB or so
for alpine.

If you depend on additional system libs that can add quite a bit of weight, depending on what they are. You can also double check to see which Erlang applications are being included, it should only be the subset your application depends on, not all of them.

A medium-sized application would probably depend on enough Erlang libs to hit that limit before any code from their own application was counted. A simple test project of mine hits 23mb with just libs, and here are the ones included in the release:


asn1-5.0/
certifi-1.1.0/
combine-0.9.6/
compiler-7.1/
crypto-4.0/
elixir-1.4.5/
gettext-0.13.1/
hackney-1.8.0/
idna-4.0.0/
iex-1.4.5/
kernel-5.3/
logger-1.4.5/
metrics-1.0.1/
mimerl-1.0.2/
poison-3.1.0/
public_key-1.4.1/
sasl-3.0.4/
ssl-8.2/
ssl_verify_fun-1.1.1/
stdlib-3.4/
test-0.1.0/
timex-3.1.13/
tzdata-0.5.12/

So either your app was extremely small or didn’t depend on much - in any case, I suspect most real-world applications will be much larger than 18mb. ERTS itself is only like 6mb, and anything in priv of your application or your dependencies will also be included.

I strongly recommend not stripping debug info - those cases you mentioned are some of the most powerful capabilities of the runtime, and are one of the reasons why OTP is such a powerful tool operationally. You are trading a small amount of space for a significant amount of functionality, to me it’s not even close to worth it to sacrifice that.

1 Like

So either your app was extremely small or didn’t depend on much - in any case,
I suspect most real-world applications will be much larger than 18mb. ERTS
itself is only like 6mb, and anything in priv of your application or your
dependencies will also be included.

Yes, it didn’t have many dependencies. My point was that the
release is what should take up the most space. You cannot get the docker image
smaller than the release and that the alpine linux image should have
very little overhead in terms of space.

I strongly recommend not stripping debug info - those cases you mentioned are
some of the most powerful capabilities of the runtime, and are one of the
reasons why OTP is such a powerful tool operationally. You are trading a small
amount of space for a significant amount of functionality, to me it’s not even
close to worth it to sacrifice that.

Agreed. I don’t think the trade-off is worth it either but if you want
the smallest image possible you should know about the option.

We generally encrypt the debug info (we ship our software to clients).
In this case you need the key to be able to do any debugging but it
makes it a little bit harder to reconstruct the source code.

I’ve measured the space taken inside the container, and here’s what I’ve got.

  • here’s the root:

    docker run -it gmile/ritm:0.0.9 du / -h -d 1
    0       /dev
    4.0K    /home
    0       /proc
    4.0K    /root
    8.9M    /usr
    4.0K    /tmp
    812.0K  /bin
    80.0K   /var
    0       /sys
    1.3M    /etc
    4.0K    /run
    4.0K    /srv
    4.0K    /mnt
    220.0K  /sbin
    5.3M    /lib
    16.0K   /media
    55.7M   /app
    72.3M   /
    
  • here’s the /app folder, containing unpacked tar release:

    docker run -it gmile/ritm:0.0.9 du /app -h -d 1
    52.0K   /app/bin
    380.0K  /app/releases
    33.7M   /app/erts-8.3
    21.6M   /app/lib
    55.7M   /app
    

ERTS itself is only like 6mb

@bitwalker from what I am seeing, erts is 33.7 Mb way bigger than 6 Mb. Any ideas why so?

Upgrading the base image to bitwalker/alpine-elixir:1.4.5 (with erlang 20.0.1 underneath) shaved off another 12 Mb down to 48 Mb. After looking at the size stats, the shrink is due to erts loosing some weight:

docker run -it gmile/ritm:0.0.10 du /app -h -d 1
21.1M   /app/erts-9.0.1
52.0K   /app/bin
380.0K  /app/releases
22.2M   /app/lib
43.7M   /app

The size of erts is still way more than 6 Mb.

You’d have to show what is taking up that space in the ERTS folder - on my laptop, a release packed with ERTS included only shows 5.4mb in the erts-9.0 folder via du -hs <path>. Either there is more in the ERTS folder on Alpine which can be stripped (and I can tweak that in the Alpine images I produce if so) or it’s just bigger on Linux for some reason, but I can’t imagine why that would be.

1 Like

Here’s what I’ve got:

docker run -it gmile/ritm:0.0.10 du /app/erts-9.0.1 -h -d 1
19.3M   /app/erts-9.0.1/bin
872.0K  /app/erts-9.0.1/include
4.0K    /app/erts-9.0.1/doc
852.0K  /app/erts-9.0.1/lib
12.0K   /app/erts-9.0.1/src
4.0K    /app/erts-9.0.1/man
21.1M   /app/erts-9.0.1

Looking closer, looks like it’s erts binary that takes the majority of that space:

docker run -it gmile/ritm:0.0.10 du /app/erts-9.0.1/bin -h
19.3M   /app/erts-9.0.1/bin

Interesting, so maybe on Alpine it’s statically linked? I’ll have to review how I’m compiling Erlang in the base image, maybe I have some options set that aren’t needed.

If that’s a lot to you :sweat_smile:

Most images I usually deploy are +500MB… (not my fault, but many people tend to just take Ubuntu as a base image and add a lot of unneeded crap to it…)

Congrats for trying out multi-stage, it’s a really nice feature I’ll start to use once our CI is running the latest Docker version.

1 Like

My own alpine image built through edib with a bit of tweaking are around 23 to 40 MB depending on js and assets most of the time. With few assets, 23 MB is a classic.

1 Like

~20 MB will be the lower boundary I think (without stripping or zipping the source code; empty hello world project).
But I also think this is okay enough and mostly better than 500MB to 1GB sized images I currently see floating around in Docker based setups of some companies.

If I understand correctly, there are at least some libs that erts is linked dynamically against:

docker run -it gmile/ritm:0.0.10 ldd /app/erts-9.0.1/bin/beam.smp
        /lib/ld-musl-x86_64.so.1 (0x5556a11bd000)
        libz.so.1 => /lib/libz.so.1 (0x7f6fda905000)
        libncursesw.so.6 => /usr/lib/libncursesw.so.6 (0x7f6fda6b0000)
        libc.musl-x86_64.so.1 => /lib/ld-musl-x86_64.so.1 (0x5556a11bd000)

I didn’t know about edib, thank you!

I still need to push some PR, and tbh, the new docker multi stage support nearly kill the need for edib.

https://docs.docker.com/engine/userguide/eng-image/multistage-build/

1 Like

Yeah I dug around in my alpine-erlang build today, it looks like the bulk of the extra space in beam.smp is in the .debug_loc and .debug_info sections of the binary (~11mb) - so I assume one could strip those easily enough, but whether that’s ideal or not is a different question I suppose.

1 Like