Workaround for Docker with Erlang/OTP 21.3

erlang
docker
deployment
devops
#1

Just a quick heads up: There seems to be a bug in Erlang/OTP 21.3, which can cause some errors when making http requests. If you’re using the official Elixir images that use Erlang/OTP 21 it might affect your app, because they got rebuilt with the latest version that contains the bug.

The quickest workaround is to instead of using FROM elixir:1.7.4 in your Dockerfile, replicate the elixir image layer and point it to an earlier image, for example 21.2.7:

FROM erlang:21.2.7

# elixir expects utf8.
ENV ELIXIR_VERSION="v1.7.4" \
	LANG=C.UTF-8

RUN set -xe \
	&& ELIXIR_DOWNLOAD_URL="https://github.com/elixir-lang/elixir/archive/${ELIXIR_VERSION}.tar.gz" \
	&& ELIXIR_DOWNLOAD_SHA256="c7c87983e03a1dcf20078141a22355e88dadb26b53d3f3f98b9a9268687f9e20" \
	&& curl -fSL -o elixir-src.tar.gz $ELIXIR_DOWNLOAD_URL \
	&& echo "$ELIXIR_DOWNLOAD_SHA256  elixir-src.tar.gz" | sha256sum -c - \
	&& mkdir -p /usr/local/src/elixir \
	&& tar -xzC /usr/local/src/elixir --strip-components=1 -f elixir-src.tar.gz \
	&& rm elixir-src.tar.gz \
	&& cd /usr/local/src/elixir \
	&& make install clean

CMD ["iex"]

Posting it here, because I know it affected more than just myself, might save some of you some time.

11 Likes

GenStateMachine terminating after upgrading to 1.8
GenStateMachine terminating after upgrading to 1.8
Timeout errors with long running transactions
#2

Thanks - I got bit by this too. If you’re using Elixir 1.8.1, I made an image over here: https://hub.docker.com/r/tomtaylor/elixir (using this Dockerfile: https://github.com/poplarhq/elixir-docker-image)

2 Likes

#3

This is part of why treating specific Docker image tags as mutable is so dangerous. When I’m in front of a computer I can check if I still have the correct SHA256 image checksums for the prior version that used OTP 21.2.x, because I pin them in my FROM lines after experiences similar to this.

2 Likes

#4

The historical version of the official image that included OTP 21.2.6 can be obtained by referring to this image syntax:

elixir:1.8.1-alpine@sha256:f6a0647245a467f7affee2e635341eb40d4b9dee88467f3b14a30707dbd21308

This syntax is compatible with, among others, docker pull, docker run, FROM in a Dockerfile, and Kubernetes manifests. The checksum is output whenever you pull an image via the normal docker pull CLI, and is also available after the fact via docker inspect $imageID under the RepoDigests key.

You can bulk check your image tags and digests like this, which may catch historical copies that were untagged by a subsequent pull:

docker images elixir -q | xargs -n1 docker inspect | jq ".[] | { tag: .RepoTags[0], digest: .RepoDigests[0] }"
10 Likes

#5

Oh man, this was driving me crazy for days. Thanks.

1 Like

#6

Wir seeing errors like

2019-03-19T08:54:39+00:00 app[postgres.9192]: [DATABASE] [8-1]  sql_error_code = 00000 LOG:  PID 9135 in cancel request did not match any process
2019-03-19T08:54:40+00:00 app[postgres.9105]: [DATABASE] [9-1]  sql_error_code = 08P01 LOG:  SSL error: tlsv1 alert internal error

Seems to be the bug which is decribed here

1 Like

#7

Looks like OTP 21.3.1 was released but I can’t find release notes aside from the ones saying the ssl app was changed. I might be blind though :thinking:

Judging from the commit log though, this was fixed? Has somebody tried their luck with 21.3.1?

$ git log --oneline OTP-21.3..OTP-21.3.1
a4f71b669a (tag: OTP-21.3.1, origin/maint-21) Updated OTP version
48eedeb93d Prepare release
18940986f9 Merge branch 'ingela/ssl/recv-timeout-bug/ERL-884/ERL-883/OTP-14701' into maint-21
7b4f3319aa Merge branch 'ingela/ssl/transport-transparance/ERL-861/OTP-15679' into maint-21
d4266651c5 Merge branch 'rickard/erl_call/ERL-881/OTP-15676' into maint-21
543ca376a9 Add smoke test for erl_call
f85c6c84fa Add new api functions to ei_fake_prog
adf61f3cbf Fix timeout value when waiting for emulator start
e645b6b4a5 Fix initialization of erl_call
18e8014fe2 ssl: Cancel recv timer in all places
5a7cfb2d57 ssl: Fix transport transparancy
1 Like

#8

You need to look at either the erlang-questions or erlang-announce mailing list. Or you can just check here: http://erlang.org/download/otp_versions_tree.html.

2 Likes