Crystal is great and Amber looks nice Elias
What kind of user is it targeted at and what kind of apps does it excel at?
Crystal is great and Amber looks nice Elias
What kind of user is it targeted at and what kind of apps does it excel at?
Yes I am a staunch critic of the TechEmpower benchmarks (as a way to compare frameworks. They are likely useful as a development tool for a particular framework to find some performance bottlenecks). I have given reasons why in other posts.
They should not be used to compare frameworks. They even state so themselves. Plus the way they conduct the benchmark only makes them useful if you want to know how fast a specific framework can theoretically be with the wind in the back on a slight downhill slope. I’ve tried some of the faster frameworks in a more realistic scenario and they actually behave quite poorly and the benchmarks led me down the wrong path to even explore them at all.
In regards to which kind of app it excels at? We are hoping that it can be use for any type of Apps really, since it supports cluster, concurrency soon to add an api generator and more. I mean you can even create a “sinatra” style kind of app. So if you want go with bare-bones see https://github.com/amberframework/online-docs/pull/36 you can or if you want something more robust you can as well.
Crystal aims to be a much wider scope language (and most certainly NOT just for numbers crunching!) than elixir and I am rooting for them as I like what they are trying to achieve. I guess a certain amount of “my toys are and will always be shinier than yours” is natural but I have never found it helpful or rational with technology.
As great as Elixir and Erlang are Many highly concurrent based operations have been stable without it so there are always multiple tools for the job and ALWAYS a good thing when you have options. saying one optionwill always be the best is a strange thing for anyone who has watched technology For any amount of time to say. I already envy Crystal for their choices in web frameworks. Eliixir at this point seems married to Phoenix which I can live with but am not a huge fan of. Frankly I think If they get to 1.0 Crystal is likely to take off and bigger than Elixir.
I contributed the Elixir code to that benchmark. I do not speak for the project, but I can comment a little about it. The Brainfuck 2.0 benchmark compares the performance of a naive BF interpreter that is structured in a certain way written in different programming languages. This is unlike what you see in, say, The Benchmarks Game where the participants end up with wildly different code in different languages. It ensures a certain kind of parity between the languages (no performance optimizations unrelated to the choice of language are allowed), but the structure fits some languages better than others. The result is that the Elixir implementation is slower than it could be. If you compiled BF programs to Elixir modules, which is what I would call an idiomatic solution, and applied a few simple optimizations, you’d end up with a considerably faster BF implementation. Source-to-source compilation gets you decent performance even on platforms slower than BEAM (see, e.g., my example in Tcl).
All that said, Crystal is faster than Elixir at single-threaded computational workloads. My own experience is that I tried doing image processing in both Crystal and Elixir (parse an uncompressed image file, apply a transformation to the pixels, save the result) and found Crystal to be about an order of magnitude faster at it. However, Crystal’s Fibers (green threads) are N:1 for now, so Elixir has the upper hand at parallelism.
Egads! I just looked into that, the code in a variety of languages is just horrible! Like the C++ code, like what on earth is going on with the
Op structure in the BF2 listing, it is fat, like it’s going to take up dozens of bytes in the best case, each, in addition to a growing vector, plus there is no point to encode ‘yet another’ vector inside a vector when indices would work far better. In addition the C++ code takes up a lot more space per element because of the fat
Op structure compared to most of the other languages, seemingly for no reason! And this is just a couple of the many C++ issues I see, and I’ve not even mentioned the other languages… o.O
That’s precisely where I would like to use Crystal. I would use Elixir as the core of the app - because the app needs to do lots and lots of things at the same time - and languages like Crystal for specific things where they excel (such as image manipulation)… but only when actually needed.
I like the analogy where Elixir is the brains (of an app) and other languages can be organs. The brain ties everything together, communicating instructions, receiving messages, interpreting them and then passing on other commands, etc.
The great thing is you can do most things with Elixir/Erlang (with common libraries such as ImageMagick) and only drop into other languages for very specific things when you actually need to; the core of your app, or the overall system, is unlikely to change. It’s one of the reasons I love the Replaceable Component Architecture (and love Elixir for making it so easy).
I’ve written about Phoenix’s super-fast view rendering, but IMO the killer performance feature of any BEAM-based web application is cheap processes and pre-emptive multitasking.
I don’t know how Crystal handles concurrency, but if you’re running (say) Rails with 10 Unicorn processes and you get 10 expensive requests, all your processes are now busy. Any additional requests must get in line to be processed, and those users will see the site as having very high latency. If it takes 3 seconds before a request even gets to Rails, Rails may tell you that it handled that request in 200ms, but that’s only part of the story.
By contrast, every request to Cowboy gets its own BEAM process immediately, and because the BEAM uses preemptive multitasking (just like your operating system does), a process with a lot of work to do has very little effect on ones that don’t. I’ve explained how this works in a blog post, “Concurrency vs Parallelism and the Erlang Advantage”. Also, Saša Jurić’s talk “Solid Ground” has a very nice demo.
I would use Elixir as the core of the app - because the app needs to do lots and lots of things at the same time - and languages like Crystal for specific things where they excel (such as image manipulation)… but only when actually needed.
This seems like a great strategy: the BEAM in charge at the top level, delegating to languages with more raw speed for specific tasks.
A typical bad benchmark, it just tests more or less bad ports of an algorithm.
One might even think that there is some intended outcome coded into the ports … even ruby could be faster than ASM … if you try hard enough to get a real bad port to ASM.
If one would realy try to make a benchmark fast, he would use all tricks the language provides … why on earth should an implementation in C++ be ever slower than in C ? They may even be identical, if C++ does not help to make it faster …
Yeah this is something I’ve mentioned a lot. There is absolutely no reason for any C++ code to be slower than any C code, at worst it will be equal, at best C++ will be able to generate MUCH better code mostly due to templates.
I just discovered the TechEmpower benchmark , as I was considering a new (for me) framework / langauge for a personal project . Basically debating between sticking with Ruby ( and Rails or Padrino), or trying our Elixir / Phoenix or Crystal / Amber.
The benchmark empressed me enough that I wanted to see how well it would perform for my one of my use cases: large number of requests to a JSON endpoint that pulls a (random) row or a set of 10 from MariaDB table of half a million rows.
My conclusion - real life application is way different from a benchmark.
Also for my case I needed a full framework not a micro framework ( hence not Sinatra or similar). Because of the number of different features I need that I would need to implement - I would end up with a fairly large app anyways, so might as well save some time.
I ran wrk2 against all setups .
So bottom line - Amber was only about 3x times faster than Rails 5.2 . Padrino performed about the same as Rails. Phoenix was about in about the middle of the pack. Once I added a bit more juice to my test VM, Phoenix actually was able to scale up much better and overtake completion.
So my conclusion here is that while Rails is slowest of the bunch - it’s well within my requirements of being able to handle ~200 req per second under 100ms response times on a low-end VM on digital ocean or vultr.
However as I was going through these benchmarks I realized that I will need a lot of libraries that I’m dont know if Crystal / Elixir have, and I know very well how to optimize hosting for Rails, but it took me a bit of time to do it for Elixir and Amber ( and chances are I still didn’t do it right, since I’m still learning ). Also it has a massive advantage of a huge number of gems that’s will save me time, as a one-man side project (Crystal and Elixir also have a growing ecosystem, but’s it not there yet, it seems?)
I I’m still considering building out an simple api in Phoenix or Amber just for the fun of learning either one , and building Rails/Elm frontend for it .
My point here is that those benchmarks are not very useful, because they don’t take into consideration very important real life considerations, requirements and constrains
This is exactly how one should go about answering the question of “What should I use?”: Well done!
With regards to Crystal I think it seems like a much more reasonable way going forward than trying to make Ruby fast. I do think that they need a clear way to move entire gems into that performance space, though, and I’m not sure (from my fairly basic research) this is something they actually are aiming to do. It seems like a mistake to try to explicitly fix a bunch of Ruby problems and try to explicitly be a “fast Ruby” but then not have an efficient transition story for code.
Thank you for the compliment. I really want to spend some time learning either Phoenix or Amber, but I only have 2 months for this project. So I’m still debating which way to go ( till tomorrow). But that’s not something anybody but me can answer
Conversely, when I was doing my tests, I was actually able to move most of my model/ migration/controller code with very little modification from Rails to Amber, just needed to add types to variables and several functions. Which was a pleasant surprise I might say? Of course, that was a trivial part of the app, which is kind of designed to be rails-esque, activerecord-ish.
Wake me up when they’ve cottoned on to the importance of immutability in Crystal++…
I’m obviously biased in my opinions, so please keep that in mind.
But for me, one of the biggest points after making sure performance is “good enough”, is looking at the backwards-compatibility story. And I will argue, Elixir has the best story. Elixir and Phoenix both had their first stable release about 3 years ago (Elixir in May of 2015, Phoenix in August). During that time they both kept almost perfect backwards compatibility (modulo some bug fixes and security issues).
Crystal was supposed to reach 1.0 in 2017, but from the looks of it, it’s still very far away from this goal even now. Breaking changes are relatively frequent (less so recently, though).
Rails is generally introducing breaking changes in their every x.1 release - this means 2 times in the last 3 years, with the next release which is not backwards-compatible right around the corner. Ruby is also not adhering strictly to SemVer and introduced couple minor breaking changes in the point releases.
From my experience with Rails and Ruby, every version bump of Rails or Ruby is a huge endeavour - you never know what might happen and you need to test extensively. On the other hand, I never had any issue upgrading either Elixir, Erlang or Phoenix version. It’s a huge advantage not many people talk about.
In other fields maybe, since Crystal is a bare metal language. But I believe as for Web framework, though is not fastest, Phoenix is superior compared to most of the others because distributing and fault tolerance (BEAM’s nature). And these two factors are very important for web servers, as well as very hard for programmer to make them right if it’s not under the infrastructure.
Elixir has a very important advantage no Oracle IP.
First of all, Crystal is NOT, absolutely NOT parallel. It is concurrent, but not parallel. In other words it is single-core only, you will never ever get use of more that one core on a system without sharding it out like nodejs does.
So at it’s basic setup, Phoenix will run circles around Amber on multi-core systems, however Amber will be faster on single-core systems.
Rails is absolutely abysmal in my tests, summary of just raw throughput:
(Amber is not in the above results because it’s code is bugged but it’s close to the other crystal frameworks).
Amber is a LOT faster than Rails on raw throughput, but as always the user code will affect a lot.
Yep, this is because Crystal the language (and thus no framework on it) is not multi-core capable.
Eh, I don’t know about Crystal, but Elixir is not a one-man project, it is backed by a company and has a lot of authors at this point (with a BDFL though), in addition, as you can see on recent stack overflow benchmarks, Elixir is one of the fastest growing languages out and already in the top 20 of server usage.
Exactly, benchmarks are not useful for showing real information at all. I think micro-benchmarks (like my above linked raw throughput testing) can be useful to show the implicit overhead, but large benchmarks are near-useless.
You can get things built yourself fast in phoenix though, while having a far far faster base speed, and being significantly easier to reason about.
Yeah, I really really don’t like the base syntax of Crystal… Even Rust does default-immutable… >.>
Do not underestimate the importance of this!!!
‘Mostly’. There’s a reason I don’t call OCaml a bare-metal-language even though it’s near C speed and compiles to direct machine code, and Crystal has the same reason, it has a GC, and unlike OCaml’s GC, Crystal’s GC is the C’based HB Collector, really basic and not terribly fast, I.E. you get ‘non-base-metal’ random costs because of collections. In addition the GC only manages memory resource, not other resources, where if the language was better designed and could manage all resources in a scoped manner (like Rust does) then it wouldn’t need a GC and it would have a unified syntax for all resources. As it stands you either have to pass callbacks (that do get inlined admittedly) and do all the work inside that (think the
with command in Python) or you have to explicitly and manually call the reclamation functions for the resource (and don’t forget that or it ‘might’ get reclaimed by the finalizer if a finalizer was made for it, and even then that will only be ‘sometime later’).
Phoenix also naturally scales across machines as well.
Crystal has Oracle IP?! o.O
Dah for some reason thought it compiles to JVM
I agree with @OvermindDL1 Elixir/Phoenix r than Crystal/Amber on multi-core, can’t compete with that at the moment, even though it is possible to use Amber in cluster mode in linux VMs.
Crystal and Amber has a lot to do to catch up in many things. For instance one of the things that makes Amber framework even slower is the current available DB drivers for the crystal language could be improved.
Benchmarks, is just a point of reference that should be considered but not taken too seriously in IMO.
Anyways I love Ruby, Elixir and Crystal they all great languages for developer happiness