ericmj

ericmj

Elixir Core Team

The Hex team has been working on building our own Elixir and Erlang Docker images. We think they can be useful for the general community so I am sharing them in this post. The images are built by Bob which is the build service that provides the builds when you use Travis CI, GitHub CI, asdf, and other tools and services.

The reason we are building our own Docker images is because we have identified and run into a few issues with the existing image offerings that we are hoping to address with these images.

Issues with existing offerings

To sum it up:

  1. Image tags are not immutable
  2. Delay in availability of new versions
  3. Cannot pick the combination of versions you want

Our main issue is that image tags are not immutable. If you pull the image elixir:1.9.4 you do not know which Erlang version or OS version you are getting. This is specially a problem when the tag the image points to changes to update the Erlang version. Recently Erlang introduced a breaking change in a minor version that made it an error when you started a client side SSL connection with the honor_cipher_order option, the Elixir image was updated to the new Erlang version which caused hackney to break and our deploys stopped working without any changes to either our dependencies or our Dockerfile.

The images we build with Bob are immutable and the versions used are explicit. An example of an Elixir image tag is 1.10.0-erlang-22.2.4-alpine-3.11.3, here we can see that the Elixir, Erlang and Alpine versions are explicit and there is no reason to update any versions behind the scenes.

Sometimes there are delays in the availability of images for new releases. As I am writing this the new images for Elixir 1.10.0 are not yet available on the official Docker images even though 1.10.0 was released 3 days ago. The images from Bob are built automatically and should be available within 30 minutes of tagged releases.

Our final issue has been that we cannot pick the exact versions we want. From the official images I can install Elixir 1.9.4 but I have no idea what Erlang or OS version will be used. Sometimes we want to use the latest features from Erlang or stay on an older version for stability but this is not an option. The images from Bob are built with all compatible versions of the underlying OS, Erlang, and Elixir.

Possible downsides with Bob’s images

First off, they are currently experimental. This means you should take care before running them in production. We are running them in production for all Hex services without issue so far. While they are experimental the immutability guarantee of image tags may not hold if we find issues that require such changes.

We currently only provide images on the latest Alpine versions. In the future we may build on more systems.

Since we build for all compatible OS, Erlang, and Elixir versions we produce many, many images that use up disk spaces. We haven’t run into any limitations on DockerHub yet, but it is possible we do in the future. As far as I can tell there are no documented limitations.

To build the images faster we use the pre-compiled versions of Erlang and Elixir that Bob already provides. Because of this the original Makefiles are not available so we cannot use make install to install files in the correct locations. Currently we install Elixir and Erlang in the root of the filesystem at /elixir and /erlang respectively. This is obviously not ideal so if anyone has a solution to this problem we are all ears.

Links

To read more about Bob’s docker images go to our README: GitHub - hexpm/bob: The Builder · GitHub. Links to the DockerHub repositories can be found here: hexpm/erlang - Docker Image and here: hexpm/elixir - Docker Image.

Showing Posts 1 to 10

Phillipp

Phillipp

Nice! How about versions for ARM, so we can easily use it on a Raspberry Pi?

ericmj

ericmj OP

Elixir Core Team

We need to make sure the current images work correctly first but I think we can support ARM in the feature. We don’t have ARM builds of OTP so the first step would be to add cross-compilation to ARM in the Bob OTP builds.

Phillipp

Phillipp

Cool. I am looking forward to it. It’s always a real struggle if you want to use Docker on a Raspberry Pi because many things are just not available for ARM. I think having this sorted out for Elixir will be very helpful.

entone

entone

The “not immutable” image tags have bitten me/us several times. Excited for officially supported community versions. Great work!

mhs

mhs

Any plans for Debian (stable) slim builds? [buster, at time of this post]

I like Alpine but I’ve actually switched back to buster-slim over subtle concerns around glibc / musl and DNS.

ericmj

ericmj OP

Elixir Core Team

We want to support more OSes and CPU architectures when we know that the images are stable. The first step would be to build OTP against those architectures and OSes. Here you can find our existing linux builds [1], any contributions to support more are welcome.

[1] bob/priv/scripts/otp at main · hexpm/bob · GitHub

foggy

foggy

This is great news. We’ve been internally pinning the official Docker images ever since the incident with the ssl dep and erlang broke all of our stuff (twice). This will likely save us that trouble in the long run.

sorentwo

sorentwo

Oban Core Team

Having been stung by the mutable erlang changes (hackney ssl), mutable alpine version (libvips mismatch), and an annoying lag between version release and docker availability; I’m all for this.

I trust the hex team to provide timely and consistent builds more than I trust the unknown (to me) authors of the official image.

Thanks!

Exadra37

Exadra37

Oh it seems that I was not the only one in need of immutable images :slight_smile:

╭─exadra37@thinkpap-l460 ~/Developer/Projects/elixir-docker-stack  ‹dev-new-approach› 
╰─➤  sudo docker image ls | grep -i erlang
exadra37/phoenix-dev       1.4.11_elixir-1.9.4_erlang-22.2.1_git   23d45190fd25        3 weeks ago         814MB
exadra37/elixir-dev        1.9.4_erlang-22.2.1_git                 98f3ef6b30ac        3 weeks ago         713MB
exadra37/erlang-dev        22.2.1_git                              96c72e47095b        3 weeks ago         707MB
exadra37/phoenix-dev       1.4.10_elixir-1.9.4_erlang-22.1.6_git   46d86f30f487        3 weeks ago         813MB
exadra37/elixir-dev        1.9.4_erlang-22.1.6_git                 14df71ee5f57        3 weeks ago         712MB
exadra37/erlang-dev        22.1.6_git                              401ebb0c0162        3 weeks ago         706MB

But in my case I have built them as part of an Elixir Docker Stack that automates a lot of stuff for my developer needs, and I compile Erlang, Elixir and Phoenix directly from Github, and I am based on Debian, mainly because of Observer not working in some Alpine based images.

Thanks for this work. I need to try it out, and compare it with my approach, because Alpine based images have not worked for me.

Exadra37

Exadra37

It’s not about Docker, but about how you write the Dockerfile.

If you want a truly immutable docker image until the OS level, then you need to pin everything you install with apt or whatsoever package manager you use to install things in the OS.

But if you want to extend another docker image and have it pinned, you cannot use the tag, instead you need to use the digest hash of it. Read more in the official docs:

Pull an image by digest (immutable identifier)

So far, you’ve pulled images by their name (and “tag”). Using names and tags is a convenient way to work with images. When using tags, you can docker pull an image again to make sure you have the most up-to-date version of that image. For example, docker pull ubuntu:14.04 pulls the latest version of the Ubuntu 14.04 image.

In some cases you don’t want images to be updated to newer versions, but prefer to use a fixed version of an image. Docker enables you to pull an image by its digest . When pulling an image by digest, you specify exactly which version of an image to pull. Doing so, allows you to “pin” an image to that version, and guarantee that the image you’re using is always the same.

Dockerfile Example from docs:

FROM ubuntu@sha256:45b23dee08af5e43a7fea6c4cf9c25ccf269ee113168c19722f87876677c5cb2

So no matter how many years we go into the future, the image will be always exactly the same you download since it was built. It’s like a Git commit, if it changes the hash is not the same anymore.

Where Next? Top

Trending in Discussions Top

AstonJ
As the title says, please share what you’ve been up to with Elixir. Whether that’s been learning it, looking into it, making stuff with i...
2977 92995 915
New
AstonJ
The obligatory hello world thread! Who are you and where are you from? :stuck_out_tongue:
4616 55835 594
New
caslu
I want to open this thread for you all to discuss and help those who really like Ash but are still hesitant to use it in a real project. ...
New
Herve37
We’re evaluating API mocking tools for OpenAPI-based projects and would love to hear what other teams are using. We’re particularly inte...
New
matt-savvy
Is there a word for the ~> symbol used in Version strings? Do you also just call it a Squiggle Arrow™ ?!
New
GES233
I’m posting this in response to Jose’s recent tweet (Cr. link) : People are sleeping on Elixir for a coding harness: Hot-code swappi...
New
nseaSeb
AcmeScript — Writing JS hooks as if I were still using Elixir I’ve been having fun building a little something over the last few days: Ac...
New

Other Trending Topics Top

garrison
Hobbes is a low-level distributed database for the Elixir programming language. Hobbes provides a simple, safe, and scalable storage lay...
New
mcass19
ExRatatui lets you cook up rich terminal UIs in Elixir, powered by Rust’s ratatui via Rustler NIFs. Build interactive terminal applicatio...
New
Damirados
Hello everyone. After busy few months I am happy to announce v0.1.0 of Emerge & Solve. They are GUI (Emerge) and State management (S...
New
netoum
Corex is an accessible, unstyled UI component library for Phoenix that integrates Zag.js state machines using Vanilla JavaScript and Live...
New
wintermeyer
There are three potential reasons for members of this forum to have a look at https://vutuv.de You are tired or annoyed of LinkedIn. Yo...
New
webofbits
Aludel - LLM Evaluation Workbench Aludel is an embeddable Phoenix LiveView dashboard for evaluating and comparing LLM prompts across mult...
New

We're in Beta

About us Mission Statement

Options

Thread Display Mode




Thread Preview

Skip Thread Previews