What would be a good programming language to speed up Elixir?

Questions / Help
21

The OP was about performance. Go is by far one of the fastest performing languages out there.
That’s not debatable, that’s documented.

With that in mind, you can be critical of syntax, but I think that’s more of a “religion” argument than it is about factual data and what the OP had asked about.

1 Like
22

Its very hard to interact with Go natively (C-Port or NIF), many boilerplating that were done in C by a handfull of macros has to be done individually per go-package or targeted BEAM-module, just because there are no mechanics for abstracting it away.

This makes it unlikely to be a candidate to use it alongside erlang/elixir/lfe/alpaca/BEAM in my eyes.

6 Likes
23

Yes, but for example Rust is faster, and have much nicer semantics and is way more declarative than Go. So if it’s really all about performance you’d probably go with pure C and some pointer magic, and if you want way safer and/or more declarative you’d choose Rust, OCaml or Haskell.

3 Likes
24

I think people’s perception of go depends on where they come from.

  • If they come from C, well then anything feels safer and easier to use so
    they are happy.
  • If they come from a dynamic OO language like python or ruby then they
    think it is so fast and that the type system is the best thing since
    sliced bread.
  • If they come from Java or C# they think the tooling and speed of
    developing is amazing.
  • If they come from a dynamic functional language (like elixir) then golang
    feels shaky thanks to mutability everywhere.
  • If you come from ML based language the type system is just not good enough.

I think go is going to lsat because it got 1 crucial thing right. It is
super-easy to get started and write imperative code. That is why php was
(is?) king of web. Anyone can get going and just write code, no matter
how crappy the code is or the language is (and to be fair go is quite a
big step up from php ;)).

For me, golang feels just like a dynamic OO system. The type system
doesn’t give me enough help to rely on it and you still gets heaps of
odd run-time behaviour that you do with python or ruby. Simply I don’t
feel confident in the code I write in go.

5 Likes
25

Sounds like the “Worse is Better” phenomenon (why should I be surprised - Go has C heritage).

2 Likes
26

The development of Erlport is in coma since 2 years

27

Here is one example why choosing a ‘quick’ language to speed up your application is sometimes touchy to say: https://mail.mozilla.org/pipermail/heka/2016-May/001059.html

28

I think this is the correct answer IMO. Go is fast and from a concurrency standpoint it shares a lot with Elixir / Erlang in spirit…but it’s not for everyone.

The things that really write it off from a compatibility standpoint would be the real disqualifier.

To the original question though, it seems like there are really two things being asked: speed up with elixir vs number crunching.

It seems like Rust/Rustler is the answer on the “speed up” side but for number crunching I think you’d have to qualify about how heavy we are talking. Even though I haven’t used them, everything I read seems to indicate that Python has a lot of well played tooling for data analysis and the same goes for the machine learning side of things. If the crunching needs to be directly attached to Elixir, Rust is probably the answer. If it doesn’t, it seems like there’s a strong case to be made for Python.

1 Like
29

There is no reason for more development. Ports are very well set. :wink:

Python has a lot of tools to munge data via ‘operations’ passed to BLAS and so forth. However that only works if you are not marshalling the data between Python and the backend a lot (similar to marshalling data to/from NIF’s on the BEAM). You could indeed do something similar via NIFs and Rust as you could with a Python backend, but with one less layer of marshalling. ^.^

EDIT: You can always use Terra too. It has the full power of LLVM underneath it (same as C++ and Rust) and you could write, hmm, ‘kernels’ in Lua to run whatever functions you want. ^.^

Heck, you could go the full microkernel route via OpenCL and process data massively on CPU (and GPU and others) too. I think there is a BEAM library for OpenCL even? :slight_smile:

Huh, he seems to be a bit against using channels in Go. Interesting…

3 Likes
30

I’d like to learn to create NIF’s in OCaml. Do you know any good learning resources or would you like to share some example code? Or should I go with rustler instead?

2 Likes
31

I recommend Rustler. If you use OCaml you need to know the ‘tricks’ to not accidentally pass managed memory across the interface or so and deal with them when you do, that’s a lot to learn to do it right (though if you already know OCaml that well then I can teach the NIF bindings easily enough). Absolutely use Rust instead.

8 Likes
32

One more general thing that I’d like to share with whomever it might concern: I’ve made the mistake in the past to think that something would be sped up by doing it in Rust where it turned out that the actual work that was performed in Rust was not worth it. There is a measurable overhead in context switching:

  • Reading out Elixir datatypes on the Rust side often involves some parsing with many pointer jumps, since that is how the Elixir/Erlang datastructures are represented under the hood.
  • If you want your results to be usable in Elixir, you often need to convert your Rust datatypes back to e.g. an immutable Erlang datastructure.
  • I believe the runtime performs some work to make sure the BEAMworld matches what the NIFside would expect. (Note that this is limited; it is very easy to break everything when writing a NIF by e.g. subtly corrupting memory, which is one of the reasons why writing NIFs in Rust is nicer than C).

In these situations, either write a more clever algorithm in Erlang/Elixir, or restructure your algorithm such that more work is done per call to the NIF.

And as always: Measure the difference in performance, to be sure it is actually worth it :smiley:.

6 Likes
33

Also this post claims that ports are fast enough esp. when batching, so perhaps its worth a shot

3 Likes
34

Interesting data - port communication is quite a bit faster than I expected to be honest. I assumed it was best for infrequent and coarse grained messages but that is really not the case.

35

Port communication is very fast because that’s what most programs use to communicate in one form or another, so it is extremely optimized in the linux kernel (in many cases it might not even cause a kernel context switch thanks to batching!).