Beaver - a LLVM/MLIR Toolkit in Elixir and Zig

Why is it called Beaver?

Beaver is an umbrella species increase biodiversity. We hope this project could enable other compilers and applications in the way a beaver pond becomes the habitat of many other creatures. Many Elixir projects also use animal names as their package names and it is often about raising awareness of endangered species. To read more about why beavers are important to our planet, check out this National Geographic article.

Quick introduction

Beaver is essentially LLVM/MLIR on Erlang/Elixir. It is kind of interesting to see a crossover of two well established communities and four sub-communities. Here are some brief information about each of them.

For Erlang/Elixir forks

  • Explain this MLIR thing to me in one sentenceMLIR is like the HTTP for compilers. You can build your own compiler with it or use it to “talk” to other compilers with MLIR support.
  • Check out the home page of MLIR.
16 Likes

This is extremely cool.

With this library, one could make a crystal for elixir in elixir! As I believe that is how crystal was created – just ruby and its llvm gem.

Awesome contribution to the elixir ecosystem. Would be cool to try and make that “crystal but elixir” with this – though one would have to think about their strategy at “emulating” parts of the beam (like GenServer) while not having the virtual machine, or decide which parts of erlang to discard.

Of course, one would need to also evaluate the why of a crystal-but-elixir language. Crystal itself is still carving out its niche and I don’t see a good reason atm beyond vague possibilities. :thinking:

I was thinking about similar question. I think it might be very useful if we could implement the pre-emptive behavior of Erlang scheduler with a compiler rather than a runtime.

if I understand this correctly, it’s the other way. It’s compiling LLVM MIR to BEAM? So you could make a ruby for BEAM (or a python for BEAM)? Am I understanding this correctly?

I saw this in the readme under goals:

Compile Elixir to native/WASM/GPU with the help from MLIR.

2 Likes

Yes you are correct, the other way is also possible, you can import a piece of LLVM/MLIR from another project and use walker to transform it to Elixir AST or one kind of lower level BEAM IRs.
To summarize these two typical use cases:

  1. Elixir AST → MLIR → LLVM → native/GPU
  2. Ruby/Python → LLVM/MLIR → Elixir AST/BEAM IR
1 Like

Does the AST to MLIR bit exist yet?

There is an example to compile a minimum control flow in Elixir: beaver/cf_test.exs at main · beaver-lodge/beaver · GitHub

With the same idea, it is possible to implement more advanced transformation like a GPU DSL in Elixir.

1 Like

I see one of the goals is:

  • Compile Elixir to native/WASM/GPU with the help from MLIR.

What’s the current plan/roadmap on this? Would this fall in the same category as crystal-for-elixir basically, as I think LLVM has targets for WASM too?

I’m curious of how one would take advantage of this second use-case.

No concrete plan at this moment. Here are some of the things around this I want to implement or has some experiments but don’t have enough knowledge/experience to make it actually work:

  • Represent BEAM SSA as an MLIR dialect
  • Implement native codegen as a Elixir protocol
  • Hook some part of Elixir compiler by hot-reload and getting the BEAM SSA

As for the LLVM support for WASM, my understanding at this point is that WASM support in LLVM is not ideal enough to make it really production-ready, you need to generate WASM by yourself if you really want the final WASM produced really small and efficient.

It is mostly meaningless. I listed it to clarify the different use cases.