Minimizing OTP release size

I’m pretty sure the majority of the size of Elixir will be the Unicode modules. The new compiler in OTP 22 might be actually better at compiling things like that, which might reduce the size somewhat. The issue is even bigger since now the Erlang’s stdlib contains another copy of unicode too.

One more thing you could try is compiling everything without line info (the option to compiler is no_line_info). This might create more opportunities for compiler passes that merge similar code together if they only differ by a line number, but that’s kind of a long shot.

I’m wondering if incremental updates would help you as well. I often update my releases not by uploading the whole thing, but by using rsync to just upload what’s changed.

Actually, the pass that removes line information if passed the no_line_info flag runs after the pass that tries to share identical code between branches, so it wouldn’t change anything unfortunately.

Yes. It is worth noting that Elixir v1.8 will be slightly smaller on this aspect though because we no longer ship with the normalization behaviour (the Erlang implementation was smaller and faster).

However, if you look at each individual .beam file, it is most likely that the sizes are coming from the Dbgi (ast) and Docs chunks, and they are both optional, so that’s what I would consider removing first. I wouldn’t be surprised if it reduces the size to half.

I believe stripping the release in distillery should be removing data from both the dbgi and docs chunks already, shouldn’t it?

The ebin of my local Elixir checkout is 4.6MB so I am assuming the ones above have not been pruned.

You’re right. Debug info isn’t being stripped from the .beam files in the firmware. Let me figure out why that isn’t happening and post an update on sizes.

Definitely. I’m very interested incremental updates. It’s a little more work for me to add that to Nerves, though, so my initial goal is to cleanup my project’s firmware releases and see where things stand. (Apologies to people making web kiosks with Nerves - I’m aware that incremental updates are the only way to truly reduce your firmware sizes.)

Here’s the update:

1. I made distillery a runtime: false dependency. This also removes the need for mix to be in the release. This dropped the size by 4.5%, but it felt good to not include the build tools. I’ll obviously need to reconsider if we start using the config providers
2. Stripping the .beam files trimmed the firmware by 35%! Everything (Linux kernel, beam.smp, drivers, wifi firmware, bootloaders, and the OTP release) are now in a 15.2 MB package.
3. @mobileoverlord had to make some PRs to Distillery and Shoehorn, so these numbers aren’t easy to reproduce yet. They also include unreleased updates to nerves_system_br.

I’m not out of the woods with size, but I’m going to savor this improvement over the Thanksgiving holiday here. I also want to see if I can live with the stripped beams.

Here are some stats:

After stripping the beams:

$ du -ks * | sort -nr
1632   elixir-1.7.4
1160   stdlib-3.6
668     kernel-6.1
512     compiler-7.2.7
356     asn1-5.0.7
312     ssl-9.0.3
284     ssh-4.7.1
264     public_key-1.6.3
160     iex-1.7.4
152     runtime_tools-1.13.1
148     sasl-3.2.1
132     crypto-4.3.3
104     socket-0.3.13
100     nerves_runtime-0.8.0

Post strip, here’s example contents of a .beam file (this one is the current winner on size in the Elixir app)

iex(9)> :beam_lib.info('/srv/erlang/lib/elixir-1.7.4/ebin/Elixir.Base')  
[
  file: '/srv/erlang/lib/elixir-1.7.4/ebin/Elixir.Base.beam',
  module: Base,
  chunks: [
    {'Line', 20, 636},
    {'AtU8', 664, 1642},
    {'Code', 2316, 230760},
    {'StrT', 233084, 45},
    {'ImpT', 233140, 232},
    {'ExpT', 233380, 400},
    {'LitT', 233788, 218}
  ]
]

I would assume that means that line number info is still present, but it looks too small to worry about.

My comment about line info wasn’t about the fact that it in itself is large. It’s more that in modules like Base and String.Unicode there’s a large degree of code duplication that can be merged between different branches, etc. Unfortunately, in many cases this can’t be done, because there’s line info for building stacktraces that differs between those branches. Removing line info could help reduce that duplication further. Unfortunately, the compiler pass that applies the no_line_info option runs after the optimisation pass that could potentially remove the duplication.