gmile

gmile

Distributing compiled elixir packages?

This is just me daydreaming, kindly please let me know if I this is completely delusional… :wink:

Maybe a naive question, but I’ll try my luck: would it be technically feasible to implement distribution of compiled .beam files for Elixir packages, apart from distributing the source code (e.g. what we have now)? Or there’s a good reason, including a technical one, it’s rather not to be done / explored?

The key goal of this would be to reduce app compilation times, especially for big applications.

For example, right now, a particular app at my employer is taking a good 5 minutes to compile from scratch on a not particularly beefy VM on GitHub Actions (there are various strategies to improve the compilation speed there, though that’s not the point of this message).

I am wondering if it’d be feasible to bypass package compilation altogether, by downloading package’s compiled .beam files. This way all .beam files for all packages are downloaded and are ready to be bundled with the application during application compilation step.

Obviously, some packages rely on compile-time configuration options, so it may be next to impossible to compile a package in a one-size-fits-all manner. But maybe there’s a way to ship compiled .beam files for packages that most people could benefit from?

One reason against compiling I can think of is, for example, the potential difficulty of verifying that the compiled package indeed matches the source code, e.g. there’s no malicious code baked in the .beam. OS-level package managers seem to have figured that out somehow though? :thinking:

Btw, if this was (or is being) discussed / explored before or elsewhere, I’d be super grateful if you could throw a link at me! :bowing_man:

Most Liked Responses

al2o3cr

al2o3cr

There’s some concurrent discussion in this thread, though with a different set of requirements:


One of the “various strategies” is caching the built BEAM files after a mix deps.compile - if you compute the cache key based just on mix.lock (so that the cache can be used across CI builds that have the same deps) you can get essentially the same results as precompiled packages but without any of the portability / security / etc headaches.

hst337

hst337

As I said, you can distribute compiled packages using package managers. Some package managers work on all popular platform, like 0install which works on Mac, Linux and Windows or Nix which works on Mac and Linux.

I personally use Nix to build and deploy packages or docker images. Nix is functional where every package is a pure function of it’s inputs and environment. And around this functional idea there is a a smart cache for built packages in a way cache works for pure functions.

Writing Nix packages for Mix applications is pretty straightforward with tools like mix2nix plus it is officially supported by nix

gmile

gmile

Another, I believe, obstacle would be the fact that .beam files compiled under a specific version of Erlang are not guaranteed to work on machines running different (likely, previous) versions of Erlang :thinking:

Where Next?

Popular in Discussions Top

Qqwy
Looking at the stacks that existing large companies have used, WhatsApp internally uses Mnesia to store the messages, while Discord uses ...
New
MarioFlach
Hello, I want to share a project I’ve been working on for a while: https://github.com/almightycouch/gitgud Background Some time ago I ...
New
chuck
Let me start by stating an assumption: Phoenix is a great approach to building REST APIs. There are many reasons for this, but I will ass...
New
nburkley
AWS re:Invent is on at the moment with some interesting announcements. One new feature in particular is the Lambda Runtime API for AWS La...
New
AstonJ
I’ve just started the Phoenix part of the utterly brilliant online course by @pragdave. On generating the Phoenix app he uses the --no-ec...
New
IVR
Hi all, I’ve seen a number of related threads in the past, but I’d still be very curious to hear an up-to-date opinion on this topic. I...
New
boundedvariable
I am going through the kafka architecture. All the features what the kafka is providing are already in Erlang. I would like hear your opi...
New
klo
Got a question about when to concat vs. prepending items to list then reversing to achieve appending. So i know lists boil down to [1 | ...
New
RudManusachi
What configs will make sense to put to runtime.exs? – A bit of how I configure apps: I have generic configs in config/config.exs, dev...
New
Owens
Hello all, I am developing a new mobile app with Flutter frontend and Phoenix backend. The mobile app has real-time task management and c...
New

Other popular topics Top

AstonJ
Posting this to see if we can make things easier for people to get into Neovim. If you use Neovim and have a favourite distro please let ...
New
Nvim
Anybody knows a comprehensive comparison of Django and Phoenix, thanks for the help. Where are they similar? Where do they differ the m...
New
Patoshizzle
After calling mix ecto.create I get this error: 17:00:32.162 [error] GenServer #PID<0.412.0> terminating ** (Postgrex.Error) FATAL...
New
ovidiubadita
Hey all, I discovered Elixir and I love it. I always wanted to learn a functional programming and I intended to go for Haskell, but afte...
New
jerry
Good day to you all. I have been struggling to get a query involving like and ilike to work. Can anyone assist me on this, please? pro...
New
pmjoe
I have a relationship of love and hate with Elixir. Lots of things are just absolutely right, but there are some things that are kind of ...
New
freewebwithme
Using vs code and installed ElixirLS: support and debugger. And I got an error popped up on start up says Failed to run ‘elixir’ comma...
New
AstonJ
Please see the new poll here: Which code editor or IDE do you use? (Poll) (2022 Edition) It’s been a while since we first asked this, I...
208 31142 143
New
bsollish-terakeet
Credo is smart enough to check for (something like) this: assert length(the_list) == 0 with this response: Checking if an enum is empt...
New
nobody
Hi! In PHP: $_SERVER[‘SERVER_ADDR’] - in Elixir? Searched the docs for ip address and the web, no good results. Thanks!
New

We're in Beta

About us Mission Statement