Elixir synthetic performance test

Almost five years have passed, I can’t find the Ruby-Vert.x-based repo to repeat the test.

But the main point in our case was not about JVM vs. ErlangVM. It was about the easy transition of Ruby developers to some faster tech stack. Since with Vert.x, you just write Ruby, and Elixir has ruby-ish syntax.

We found out that in both cases, Ruby developers have to adjust to a new mindset. The syntax is secondary.

If one wants a raw performance, they should go entirely with C++ or Rust (according to Round 20 of TechEmpower benchmark, which is latest for the moment). Or drop some NIFs into critical parts of their app.

We have Ruby, Elixir, and Java (Spring, I believe) projects in our company. Clients come with some preset, and we never had to convince them to change the stack because of inability to scale.

If we start a new project, the choice is based on the available and planned resources.

I still feel that Elixir allows me to design better solutions in terms of architecture.

You said some lag in JVM !!!
Using which JVM GC ?

I use ZGC for really huge ram but not appear any lag !!!

We also played for some time with VertX but after a while it became obvious it’s a somewhat obscure, although very very fast, technology with a very little user base, at least they are not visible.

I’ve tried their approach to web pages generation but it is like light years from the experience that you can have with say ASP NET Core. It is faster, definitely, but not when it comes to development, when it comes to development it felt like this was just a student’s project that was enough for his barber’s pages.

There are just six guys at Red Hat working actively on this project and the rest lays on the community and I don’t think it’s working out that well. I’m just working on their https://start.vertx.io/ page it could have been done in days, we are “working” on it for months because of their workload.

I love Vert.x + RxJava …, good alternative to commercial spring boilerplate.

Thank you guys. You refilled me with sufficient amount of arguments to prepare answers for myself.

I do not know if I will continue with the benchmark, but it would be nice to shorten the performance gap a little and collect adequate measures, in addition to other aspects you mentioned.

Vert.x is a polygot implementation of the node paradigm on the JVM. You can spawn ‘verticles’ to handle a single API endpoint which all communicate through a messaging backend that extends to the clients Think node + messageMQ perhaps. They do have some good abstractions to reduce callback hell.

I played with it for awhile but in the end it was a lonely affair, with very little activity on their Google group.

It should not be the GC. GC in Erlang is per-process, so there is no stop the world event.

Oh I’m positive there’s always a workaround, even if it’s not explicitly supported by the library itself. Worse comes to worst, you can always start such activity in a separate OS process.

But the thing is that you have to know upfront whether e.g some request processing is blocking. And that becomes increasingly harder as the project becomes more complex (which IME inevitably happens for every software project other than the ones which are cancelled ). The thing is that blocking might happen unintentionally, due to a bug, or non-optimal piece of code. Not only have I seen such thing happen and paralyze the production completely, but I actually caused it myself by introducing a suboptimal code With Elixir/Erlang, such mistake is much less likely to take the whole production down, or even have observable effects on it.

Another problem with explicitly identifying the blocking code is this. How can I know that foo() is potentially blocking for a long time? To know that, I need to understand the complete stack trace of foo including my own code, as well as the code of all dependencies invoked from it. And I need to consider every possible input that can arrive to foo. And when I make my decision, it’s only based on the current code snapshot. A seemingly simple and unrelated change might break my expectations tomorrow. I’m exaggerating, yes, but it’s a thing that becomes increasingly harder to manage as the code becomes more complex.

That problem is in Elixir/Erlang non-existent. If you want to run things separately you run them in different processes. It’s as simple as that

I’m really not familiar with JVM, but I’m not surprised that there’s something like that given it’s maturity. However, libraries such as Vert.x implement additional lightweight mechanism on top of VM, and therefore the request handlers are likely not special VM entities. In fact, in event-based technologies, request handlers are usually completely anonymous.

Now given that you could have a single thread multiplexing thousands of different requests there are some questions. Can you trace the execution of a single request? Can you get info (e.g. memory usage, stack trace) of a single request handler? Can you terminate a single request handler without disturbing anything else, even if that request is blocking? If the answer is no, then the tech is nowhere near capabilities of Elixir/Erlang when it comes to analyzing and fixing a live running system.

I’d be somewhat surprised if in any case Elixir turned out to be faster. But as said, considering only the speed, and measuring it in a 15s synthetic bench is IMO not a good comparison. The question should be whether both technologies are sufficiently performant for the real problem you’re solving. If yes, then it’s perhaps time to consider other aspects of both technologies, such as e.g. fault-tolerance support If not, and assuming you invested some effort into making it faster, then I guess you need to discard the option which is not performant enough, even if that option is Elixir

Thank you, will fix that.

just after a quick look: (might contain mistakes/misunderstandings on my part)

elixir is benchmarked through a router while vertx is not.
the db query for elixir has limit 20 while the vertx has limit 10 (twice the data/serialization?).
the db pools are not identical. vertx has the default 100 limit while elixir has 20.

not sure of the impact on the results, but would be nice to remove some of the benchmark smell..

Thank you for so broad reply.

I did not tried this yet, but the case is mentioned in their documentation. To be fair there is a workaround.

I was told there is JMX for this.

Will try to measure in normal mode.

In overload case the Vertx version performs about 30% better for now. Do not want to publish the results after your arguments about the methodology. The apps are accessible and it can be verified (warm the JVM first).

My previous statement about 10% was based on Vertx Mongo against Elixir static on another machine. I rewrote the app, so it is better now.

Maybe using Elli will make it even better.

Also need to verify unhandled exception case in Vertx app.