Cannot build project with c files with hexpm docker images

I am trying to use docker images built by bob and uploaded to dockerhub in the hexpm repo in combination with gitlab runners to automate tests and thing like credo. I have exqlite in my project and it is compiled from source at the moment.

I have tried to use several images so far:

  • hexpm/elixir:1.15.5-erlang-26.0.2-ubuntu-focal-20230126
  • hexpm/elixir:1.15.5-erlang-26.0.2-debian-bullseye-20230612-slim

It seems that none of these images contains the utilities like make to build from source. I was able to fix this by installing build-essentials manually, however I am not a big fan of this solution.

Is this the expected behavior? Shouldn’t these images contain all the prerequisites for building elixir projects?

No, they contain the minimal code required to have erlang and elixir run on top of their base OS images. They don’t contain build tooling by design.

There seems to be some difference between docker images hosted in elixir dockerhub repo and hexpm ones. I only started to get this problem after switching to hexpm.

Yes. The elixir ones started as community contributions and are now maintained by the EEF, the hexpm ones are build by hex bob automation.

But really you don’t want build tooling in docker images, which are meant to run in production and that’s what the hexpm ones are for. You can use them in CI and install build tools manually or build on top of them / use other images, which come with build tools installed. I however don’t think build tools installed is a good default for docker images.

I’m not sure what would be the benefit of running images with elixir and erlang preinstalled instead of a release. Even if you take a look at Dockerfile generated by phoenix with phx.gen.release, the hexpm image is just used for building the project.

You’d be surpised how often people don’t run releases, even if it’s not best practise.

Also even if you include built tools. Where do you start, where do you stop. hexpm build for multiple OSes, so using the os default will yield different results. Using a custom subset also needs (additional) maintenance across OSes. Now make might sound simple, but then people might want to compile rustler dependencies and need rust/cargo, …. I personally find it cleaner to just add erlang/elixir to the base images and leave the rest to the users, who actually knows what they need.


Makes sense, I think the system for compilation of native files is kind of a workaround currently and could be improved greatly.

Yeah, I’d love to see zig being adopted more to compile NIFs.

Yeah, we’ve talked before about this here.

I am planning to participate with this idea on an upcoming hackaton (can’t remember the name, something overflow), basically using the same approach tailwind is using currently with a mix of the options provided by burrito.

I saw mention of exqlite and thought I introduced a bug. But for exqlite to work with your base image you will need to apt-get install dependencies. There are precompiled binaries already with the release so it shouldn’t be necessary.

This is strange, because it always compiles from source, is there some special option that needs to be passed?

I am using it from the dependency ecto_sqlite3.

I would install build-essentials, compile my stuff, then just copy the output of the compilation in another Docker stage. That’s why we have multi-stage Dockerfiles.

(Or you begin with a more dev/build friendly image and just copy stuff to the minimal hexpm container. That might be an even better idea actually.)

I wouldn’t want to reinvent the minimal images already provided. But if you’re feeling brave and independent, you can always make your own minimal image stepping on e.g. distroless or alpine/wolfie. I personally wouldn’t.

I am not building a docker image that will be used in production, this is the CI stage that executes commands like: mix test, mix credo, mix dialyzer. All what needs to be done at this stage is compile the project and execute the command.

Nor do I, the whole idea of using these images is the fact that you have the elixir and OTP with a specific version, creating custom images would defeat the entire propose of using images from hexpm.

I will just say the same thing that was marked as the correct answer, the build system for files that compile to native code is a big mess and there is absolutely no guarantee that even if build-essentials were to be installed on the docker images, it would work for majority of the projects that contain NIFs.

Welcome to “modern” computing, where a C/C++ build toolchain is still (a) required and (b) extremely difficult to have in a deterministic manner. :disappointed::grin:

That’s why people push for Zig and Rust to become the baseline tools instead. Sadly it’s taking a while.

The zig build system is very promising, in theory it should solve all these problems for c/zig files with minimal effort, namely creating a zig build file. I will attempt soon to make a elixir library that integrates this, and hopefully we will forget about make, cmake, automake forever.

Do you have a demo or an example of this happening? I will take a dive into the build tooling and see if I need to fix a bug or document the build requirements better.

I will try to recreate the setup tomorrow and come back to you.