The Go vs Elixir thread got me thinking: Would it be too hard to implement a simple mechanism for creating Go-style static app binaries from Distillery Releases?
After all, Releases already bundle all Elixir/Erlang dependencies, so they seem like an ideal starting point.
I suppose, one could come up with an elaborate solution that would implement a custom boot process for the BEAM VM and somehow load all required BEAM files into it.
But why not create a “dumb” binary that is bundled with the release data (think self-extracting archive). It could dump its contents into a temporary folder and runs the app from there. A slightly more elaborate approach would be to use a user-space filesystem (FUSE on Linux) like AppImage does.
Combined with adequate compression, this could produce binaries of around 10 MB.
As far as I know, it is already possible to create self-contained releases. They contain the runtime as well as erlang itself and therefore do not need anything installed if they are pure BEAM.
If they use NIFs they need the libraries installed.
If they open other programs, the other programs need to be installed.
Since the erlang runtime alone is already huge, I do not think this would make any sense to drive any further.
Of course can can create Releases that include ERTS, but you can’t put them in a single executable binary, can you?
I disagree. A compressed Elixir release including ERTS can be around 10 MB; looking at some CLI tools written in Go, they are definitely smaller (compressed at ~6 MB), but not by orders of magnitude
Yes, I believe this to be doable. With some changes to erts itself, it’s probably doable to just compile into a single binary and embed the modules inside of it - it already embeds the code for the “core” modules like erlang or init, etc. The primary reason why this is not happening is probably just because nobody did it yet.
I remember an erlang fork/tool thing way way back in the day that packaged everything up in a tar file that the erlang executable could ‘run’, no doubt you could pack those together too (this was why using the priv_dir call was important instead of hardcoding the path!).
There is currently no tool that will package an entire Erlang app & ERTS runtime into a single executable. Within the past 6 months there was a pretty extensive discussion on the Erlang mailing list about doing this: totally doable, probably not quite as easy as some might expect, just hasn’t been enough demand for it.
ODL: the only thing I know of packed up all the erlang BEAM files and a bit of code to run them, but not ERTS–are you thinking of something different that also packed ERTS into it?
Here is the most primitive version of a static “binary” I could come up with: A shell script that extracts its contents to tmpfs. I present to you: A fully self-contained Elixir app with just 11.8 MB: => clock.sh
It’s compiled on a recent Ubuntu, so if you’re running any recent Linux distribution, it should work.
Not exactly what OP is asking about, but for any passerby who want to glance at something they could already use today, Distillery supports self-extracting archives including a variant that cleans up after itself. I don’t believe the latter reaches for a tmpfs like @wmnnd has above, though.
Oh! How could I have missed this? It seems to do exactly what my wrapper script does, except for extracting to ./tmp instead of tmpfs. Amazing!
It’s also compressed, to it creates a file that is basically the same size as the one I posted.
After some more experimenting, here is a 64 bit binary that should truly run on any Linux system regardless of its libc version or other system dependencies. I’ve successfully tried it on Alpine and Ubuntu.
Right now, this is using the OTP build from @bitwalker’s alpine image but maybe it would be possible to do a custom build of OTP for even smaller file sizes.
Let me know if you have had a chance to try it and if it worked!
Do you think this path should be explored further?
This is definitely a killer feature of go! i would personally love to see more of this, so darn cool. Good job.
Consider the ease of deployment, I know it won’t feature rollbacks etc. but you don’t always need the entire bonus package.
Thanks for trying this!
Curious that the second app doesn’t seem to work. Could you try downloading just the contents, then cd to the clock directory and run bin/clock?
If that doesn’t work, try erts-10.0.3/bin/erlexec.
I’d be interested to see what’s going on there …
The primary reason why this is not happening is probably just because nobody did it yet.
There was at one point a version of the compiler (that Joe A. wrote IIRC), which would compile an Erlang application to a static binary. It was removed from the Erlang compiler quite some time ago (around R10 I think) because it was becoming unmaintainable. I suspect it is a non-trivial effort to reintroduce it (I did some experimenting to try and bring it back, but it was more than I could tackle with the time available to me).
Short of someone whipping up a heavily modified compiler and runtime, and getting official support for it, I don’t think we’re anywhere near golang-like static binaries. As much as I’d love that capability, I’m personally of the feeling that focusing our attention in that direction is attempting to solve the wrong problem. I certainly won’t discourage anyone from giving it a shot, but I think we have lower hanging fruit.
