I’d like to package ex_doc for a Linux distribution. Packages are usually built in isolated offline environments with all sources (and potentially their dependencies) prefetched for security reasons. Since ex_doc is using mix as build tool I’m looking into a way of doing offline vendored build just like ie cargo or go support. Can anyone help? I’ve tried doing various things like populating deps/ with tar content fetched from repo.hex.pm or even shipping whole deps/ after mix deps.get, but mix/hex still insists on getting cache from network and fails.
Can you provide more details on what you’re doing? Having deps/ exist and populated should be enough to build docs. I’m not sure what would be calling hex here in the first place.
Edit: Do you copy the mix.lock file? Because without it there would be dependency version resolution happening.
I don’t think that is completely true, when using mix, it needs these pre-requisites if I’m not mistaken (I always run these commands on CI):
mix local.rebar
mix local.hex
They seem to be installed in /elixir_install_folder/.mix/.... I suppose that if they are missing there, mix tries to fetch them from internet.
Yeah, indeed. I generally consider those part of any elixir environment. Without rebar binary nothing would know how to deal with a rebar structured dependency. Nothing should need to look at e.g. the hex package cache though if all packages are available in deps/ (usually that stuff is in ~/.hex).
1 Like
I don’t think any environments includes those archives by default, if I remember even in pre-built docker images you have to install them too, however copying a version of them should be trivial.
Yeah, you do have to install them, but one also has to install elixir. That’s what I meant with being part of an elixir environment – contrary to a project level dependency.
Yeah, it would be great if @jpalus included how he got elixir/erlang installed on that offline machine to begin with, that would have helped with diagnosing the problem.
To make it more explicit:
# prefetch sources/needed dependencies
wget https://github.com/elixir-lang/ex_doc/archive/refs/tags/v0.35.1.tar.gz
wget https://builds.hex.pm/installs/1.16.0/hex-2.1.1.ez
# build
tar xf v0.35.1.tar.gz
cd ex_doc-0.35.1
MIX_ARCHIVES=$(pwd)/archives; export MIX_ARCHIVES
mix archive.install --force ../hex-2.1.1.ez
# extract prepared deps/???
# simulate build without network access
unshare -U -n mix escript.build
The question is what to put in:
# extract prepared deps/???
So last step does not ever access network (keep in mind ~/.hex does not exist). So an equivalent to:
Go:
go mod vendor
go build -mod=vendor
Rust:
cargo vendor
cargo --offline build
1 Like
Builder machines have access to distribution packages and can install packaged erlang/elixir. They are also provided with any preprepared sources. But they don’t have access to Internet.
So you can confirm that you have hex and rebar3 archives already installed?
Yes. Hex is installed explicitly from archive (see steps) and rebar3 while installed I think is not used in this particular case.
1 Like
Sorry for confusion. Actually:
shipping whole deps/ after mix deps.get
is enough. So passing ex_doc sources, hex archive and archived deps/ (with .hex checksum files included) as inputs results in successful build without network access.
3 Likes
