Memory requirement of compiling elixir

Back in the days of 1.10.x I can compile Elixir on a 1GB memory VPS. I cannot anymore with 1.11.4, the compilation will run out of memory around unicode.ex. I know it is tough; in fact Elixir is probably the least demanding sophisticated language in this regard. However I am willing to trade off some compilation speed for the ability to do so. So my question is: is there any hope to restore this ability, maybe have a slow mode to compile certain piece of code?

2 Likes

Can you solve this via adding 4GB of swap space ?

7 Likes

That is what I use on my 1gb 1vcpu VPS. Compile times are slow but never had memory issues. I guess depends on the thing you compile too.

If it’s open I can happily try to compile on mine. Have the same elixir version as you mentioned

1 Like

Just did that. Thank you. I don’t have much disk space either, but after some deleting I made it work.

Apparently, the only file that is hard to compile in elixir is the unicode.ex, I watched memory consumption throughout the process and it is the only time the memory footprint spike up significantly. What is even more interesting is that the unicode part did not change: I mean the whole content of lib/elixir/unicode did not change between 1.10.2 and 1.11.4, while the former compilation can fit in 1GB the latter cannot. This is on the same VPS, with the same Erlang version, nothing else changed.

So it has to be some other things changed in Elixir that cause the memory consumption went up to compile the unicode library. I wonder what it is.

2 Likes

Iirc the unicode module reads some other files and does metaprogramming to get itself done, those files might have changed in the meantime.

Also, if disk is a problem and you can’t afford permanent swap space, a swap file of 1GiB preallocated that you remove after the build might suffice. Would be some annoying dance to do though on each recompilation.

Therefore: have you considered prebuild versions from Bob?

5 Likes

This is what I did. thanks

Assuming this isn’t under NDA – what set of decisions led to needing to compile Elixir in such a resource starved environment ?

Nothing special; binary packages work fine. For example, I don’t even try to compile node.js or erlang. It all boils down to why not if you can. Maybe one day I’d make local modification or make a PR to Elixir.

The other side of the question is why use a cheap VPS instead of a real computer. I have both, actually multiple of both, so I try to have the same setup everywhere to reduce maintenance overhead. It is important to know the capabilities of the least powerful of your machines.

3 Likes

Did you also upgrade Erlang? It seems like there might have been a regression from 22 or 23 and I’m wondering if this is what happened to you as well.

We had our own issues with high memory consumption during compilation and Erlang 24 (which I believe includes José’s patches) did improve the situation in our tests, though we ended up solving things a different way.

2 Likes

No. I am still using OTP 21.2.6 from Debian stable.