Should Elixir invest in becoming a truffle language for Graalvm?

-Seeing that the Graalvm offers support for many programming languages and has a guide on adding your own langauge. I’m thinking that it could open doors with respect to extending Elixir functionality by using tooling provided from another ecosystem.

-Anyone familiar with the graalvm and their technologies feel that there’s anything their ecosystem can provide us with (I read it supports already ruby/js/python/lua). Do you think Elixir can leverage using the Graalvm in any way for things that we might not use Elixir for exclusively like cli/desktop tools/big data etc (like maybe if I’d want to have a desktop tool developed in Elixir, but my use case prefers it to have a faster startup time that isn’t always an option with beam)?

-I’ve also seen recently on Techempower, es4x Javascript microframework (using graalvm) is around 36% more performant than the fastest Go microframework. I cannot say that the reason for its performance is that it runs on Graalvm instead of nodejs (it uses postgres while most of the top Go frameworks use mysql), but i feel that it probably did play a role in its performance…I’m all in on phoenix btw, but I’m not sure about the of its performance metrics on TechEmpower, I’ve read people mention that TechEmpowers metrics of Phoenix isn’t true, but I have too little knowledge to speak on that…this isn’t necessarily part of the discussion, but i feel if there is an issue with the techampower benchmark, then it should be remedied.

I don’t know if it’s possible to even do this, but I am looking forward to yours ideas/opinions on the subject.

Seen GitHub - baratharon/graal-erl

Seen GitHub - yzyzsun/truffle-erlang: Erlang implementation built on GraalVM/Truffle

It is probably an interesting project but I’d not bet my production budget on it for now. The more targets you add the more problems you have… and if you need that sort of performance why not write the code in an already supported language.

4 Likes

Good point, and i do certainly agree on using the better tool for the job at hand. It just seems to me especially with the emergence of Nx (already able to use pluggable compilers from other languages) that Elixir wants to broaden its horizon from being a pure web dev shop and maybe use Graalvm for offline tooling or something.

Edit: thanks for the repo links btw

IMO the philosophy of Nx / EXLA is to allow the control plane to control more individual pockets of exogenous technology which is not the same as swapping out BEAM for another thing.

Packaging Elixir applications for desktop distribution is a known problem and although kludgy and clunky there are working solutions.

2 Likes

Yes, its not exactly the same.
Only thing I’ve seen was bakeware; it wraps and compresses the beamvm into an executable, written in C, simple idea…but it does the job I guess.

Note that a lot (most?) of the power of Elixir lies in the pre-emptive scheduling of BEAM. GraalVM doesn’t have green threads / coroutines, just relying on OS threads. This means that you would have to practically reimplement BEAM on the JVM/GraalVM to get the same kind of guarantees that we have, or opt for something like Akka. There is some prior art (Erjang) for reimplementing but it’s not a simple task.

7 Likes

I don’t want to go offtopic, but I have to say, that Elixir is not a “pure web dev shop”.

4 Likes

I’m with you on this. I see what you’re getting at. I’m not disregarding the beam and its benefits, I’m just looking at the perspective of how we could benefit from their ecosystem in places you wouldn’t use elixir

I am wawre of that, just looking at the perspective of what its more popular for today.

One thing to keep in mind is that (last I checked) GraalVM is very memory-hungry. Strap a language on top which uses persistent data structures, and the result will eat memory quickly. Some savings will probably be found because separate GC heaps aren’t required (so no redundant copies), but I cannot guess how that will affect a typical application.

At least within the domain I work – backend systems – memory is by far the limiting factor, not CPU.

1 Like

Ah yes, thanks that is something to keep in mind…i know java is typically high on memory

Don’t let us stop you from trying to build it though! There are plenty of domains the BEAM is not suited for, and if an Elixir-on-GraalVM goes nowhere you still worked on something interesting.

1 Like

Some thoughts on this:

  • I’m unsure how much startup time reduction something like GraalVM would provide; it’s a big deal in the Ruby space because you can ahead-of-time compile Ruby files, which does reduce startup time. The BEAM already compiles to its own bytecode.

  • BEAM-on-Graal would have some of the same problems as Ruby-on-Graal, especially around native functions. TruffleRuby pulled this off by writing a C interpreter to handle running native extensions. Nowadays, you’d probably ALSO need a Rust interpreter :thinking:

  • there’s a lot of things implemented in C as part of the BEAM (processes, parts of ETS, networking, etc) that would need to be completely rewritten (or interpreted, see above) into Java. truffle-erlang pulls in Akka to do some of that.

4 Likes

To add to that, based on what @KronicDeth shared in this ElixirMix podcast on Lumen (I hope I’m not misreferencing the source) when they initially tried to get a BEAM implementation in web assembly one of the biggest performance issues was the non-existing support for multiple call stacks and immutability, as the standard was mostly evolving at that point by parties not interested in functional programming (and so the Lumen team decided that they should get involved with the committee). I’m not actually familiar with GraalVM, but I would not think it improbable that an OO-oriented VM would not have good support for what Elixir needs to make it more than a toy implementation.