Hex compiled ERTS for building ubuntu releases on any OS

Chat / Discussions
, , ,
#1

I’ve often seen “complaints” of people trying out elixir and releases that it’s difficult to setup mostly because of the need of matching up build and production server. Sure one can build the release on the production server for trying things out, but that needs erlang and elixir being installed on the production server essentially removes one of the “advertised” upsides of releases for not needing to do that. One can use docker, but not everyone is willing to fiddle with that for a quick trial.

Given hex bob does now build precompiled packages of erlang for all ubuntu lts versions there seems to be another viable path however, which I tried out quickly last week: Download those precompiled packages and extract the erts portion to build the release with. Given my understanding this should work fine for any project not needing to run NIFs or otherwise depending on actual cross compilation.

The following is a proof of concept of what I built:

This feels like it could be a great way to ease people into building releases without immediatelly forcing them to use the complex solutions if all they need is ERTS being compiled on some version of ubuntu. I only did some quick experiment with an bare phoenix installation released for ubuntu 16.04 from my mac and this seemed to work fine.

@ericmj also suggested that extracting erts out of the many docker images hex builds might be an option, which would allow for even more variety, but I feel it would likely be simpler to just build the release within an docker container in the first place.

I’m wondering what people think about this approach. I might put it together into a proper library with some more documentation, like suggesting pbkdf2_elixir for not having a C dependency, ….

4 Likes
Phoenix deployment like an uberjar
#2

Sounds interesting, although just using docker is pretty simple, too.

#3

Personally I can understand the argument, but I wouldn’t underestimate how much of a barrier docker can be outside of teams working with it anyways. Also I’m not sure how great it is on windows nowadays.

#4

Please don’t. Argon2 or Bcrypt are much better choices.

#5

I mean I can add some explanation to it, but I know that e.g. Programming Phoenix does suggest that as well, simply because of not wanting to deal with an NIF dependency and the thereby included complexities.

#6

I wonder if anyone has tried to compile nifs in application start… though I suppose that would mean that you have to have a C compiler in the production image.

#7

Yeah, I’m not sure how foolproof it would actually be. The idea of using precompiled erts is mostly meant to make “getting up and running” simple for people fresh to elixir, so the approach of having options not needing NIFs in the first place seems like the simpler solution. If one latter happens to want to use NIFs or some library needing NIFs I still think it’s better to adapter the strategy of building the release to something more elaborate.

#8

I am waiting for OpenSSL 3.0 to land and then maybe we will have KDF functions in the crypto library. However to be honest I think that “Erlang crypto library” could be statically compiled and linked against boringssl or other implementation to always provide the same, fixed, set of algorithms instead of dynamically linking and checking for what is supported in the runtime.

#9

I don’t actually feel erlangs runtime dependencies are the problematic ones. They can be solved by telling people “install XX on your server and it’ll work with erts retrieved from …”. E.g. installing libssl in production seems like a reasonable dependency for not needing to care where I build my release. Most of the voices I heard complaining were people coming from ruby or php (like myself) or something like that, where the runtime dependencies are quite a bit more severe than that.

Also usually people understand that there is a reason for the complexity when projects get more custom – with NIF based libraries, where things are to be built when creating a release. Those are the ones, which either require cross compilation or the release being built on a system matching production.

What they don’t understand is why the common case of “plain phoenix built for running on my run-of-the-mill ubuntu VPS” doesn’t have a simple solution. That’s what I’m trying to improve on with this.

I’d love to suggest Argon2 or bcrypt, but currently they fall in the category of NIF based library, which means a bunch more complexity over being able to release to ubuntu from any OS just by including the correct erts package and maybe installing the few runtime dependencies erlang has.

#10

You still need to know where you build your release. In fact, I think that statically-linked libcrypto (Erlang do not use libssl) is simpler, as you do not need to know anything about runtime system, just like you do not need to install or know anything about OS-provided pcre as Erlang uses statically linked one.

And when you add to the mix external library requirement (like libcrypto) you end with exactly that problem, as crypto is in fact NIF library, but we need not only NIF, but the external library in proper version as well.

#11

I can see that it’s being simpler, but I personally have no idea how I would get to that. Why isn’t that the default in the first place if it doesn’t have a runtime dependency?

That’s the difference between trying to solve the problem generally and saying: Here is a way to build for this exact version of ubuntu with those runtime dependencies if your project doesn’t depend on any NIFs besides the ones in erlang itself. Is there still a possibility for an breaking version mismatch between the docker images hex builds erlang in vs. the production system of someone – probably. Will it in 95% of cases – likely. I’d be happy to see solutions, which are more flexible and still easy to use, but by now I’ve not seen examples of those.

Our Sponsors

© Copyright Elixir Forum | Terms | Privacy & Cookies

Elixir Forum