gmile

gmile

Distributing compiled elixir packages?

This is just me daydreaming, kindly please let me know if I this is completely delusional… :wink:

Maybe a naive question, but I’ll try my luck: would it be technically feasible to implement distribution of compiled .beam files for Elixir packages, apart from distributing the source code (e.g. what we have now)? Or there’s a good reason, including a technical one, it’s rather not to be done / explored?

The key goal of this would be to reduce app compilation times, especially for big applications.

For example, right now, a particular app at my employer is taking a good 5 minutes to compile from scratch on a not particularly beefy VM on GitHub Actions (there are various strategies to improve the compilation speed there, though that’s not the point of this message).

I am wondering if it’d be feasible to bypass package compilation altogether, by downloading package’s compiled .beam files. This way all .beam files for all packages are downloaded and are ready to be bundled with the application during application compilation step.

Obviously, some packages rely on compile-time configuration options, so it may be next to impossible to compile a package in a one-size-fits-all manner. But maybe there’s a way to ship compiled .beam files for packages that most people could benefit from?

One reason against compiling I can think of is, for example, the potential difficulty of verifying that the compiled package indeed matches the source code, e.g. there’s no malicious code baked in the .beam. OS-level package managers seem to have figured that out somehow though? :thinking:

Btw, if this was (or is being) discussed / explored before or elsewhere, I’d be super grateful if you could throw a link at me! :bowing_man:

First Post!

hst337

hst337

Or there’s a good reason, including a technical one, it’s rather not to be done / explored?

Environment defines the compilation, because there are some compile-time inputs like MIX_ENV or other env variables, or compilation of native dependencies (like NIFs or PortDrivers) is also bound to the environment of the builder.

There are plenty of tools which make builds idempotent, isolate the environment, cache builds and define the inputs of the package. I’d suggest using nix

Most Liked

al2o3cr

al2o3cr

There’s some concurrent discussion in this thread, though with a different set of requirements:


One of the “various strategies” is caching the built BEAM files after a mix deps.compile - if you compute the cache key based just on mix.lock (so that the cache can be used across CI builds that have the same deps) you can get essentially the same results as precompiled packages but without any of the portability / security / etc headaches.

hst337

hst337

As I said, you can distribute compiled packages using package managers. Some package managers work on all popular platform, like 0install which works on Mac, Linux and Windows or Nix which works on Mac and Linux.

I personally use Nix to build and deploy packages or docker images. Nix is functional where every package is a pure function of it’s inputs and environment. And around this functional idea there is a a smart cache for built packages in a way cache works for pure functions.

Writing Nix packages for Mix applications is pretty straightforward with tools like mix2nix plus it is officially supported by nix

gmile

gmile

Another, I believe, obstacle would be the fact that .beam files compiled under a specific version of Erlang are not guaranteed to work on machines running different (likely, previous) versions of Erlang :thinking:

Last Post!

lukaszsamson

lukaszsamson

ElixirLS Core Team

There are some other approaches for distributing beams:

Where Next?

Popular in Discussions Top

acrolink
How does the two languages compare when it comes to server side application development? Any experiences or ideas? Thank you.
New
jer
I’ve been using umbrellas for a while, and generally started off (on greenfield projects at least) by isolating subapps based on clearly ...
New
PragTob
Hello everyone, I know we had quite some threads (read through lots of them) about background job processing but it remains a hotly deba...
New
RudManusachi
What configs will make sense to put to runtime.exs? – A bit of how I configure apps: I have generic configs in config/config.exs, dev...
New
crispinb
On reading dhh’s latest The One Person Framework it strikes me that Phoenix with LiveView is already pretty much this. However, never hav...
New
AstonJ
Can you believe the first professionally published Elixir book was published just 8 years ago? Since then I think we’ve seen more books f...
New
paulanthonywilson
I like Umbrella projects and pretty much always use them for personal Elixir stuff, especially Nerves things. But I don’t think this is ...
New

Other popular topics Top

rms.mrcs
Hi, I need to transform a list of numbers into a map where the keys are the indexes and the values are the original values of the list. ...
New
minhajuddin
I have seen a lot of code which picks the first element from a list using Enum.at(0) instead of List.first. Is there a reason why people ...
New
lanycrost
Hi everyone! I need implement if…else if…else condition from my elixir code, and anymore of this control flow structures not work proper...
New
ashish173
I am using Ecto timestamps with postgres, I can see the timestamps() use the :naive_dateime but for my use case I wanted to store the ti...
New
nsuchy
Hi. I’ve noticed that Windows Powershell has it’s own IEX command and you cannot access Elixir’s IEX due to the conflict. This isn’t a cr...
New
AstonJ
Seen any cool LiveView demos, sample apps or examples? Please post them here! :003:
New

We're in Beta

About us Mission Statement