This is plain false. It’s trivial to saturate all your cores with the BEAM. The overhead is minimal compared to CPU-intensive workloads and there is no bookkeeping that will ever contend with an actual CPU-bound task. As @cmkarlsson said you have a bottleneck somewhere in your system that is causing this. Even with thousands of processes being scheduled over several cores you should be able to keep your processors pinned given an actual CPU-bound, consistent workload.
gon782
josevalim
Creator of Elixir
So you are using a mostly IO bound workload to profile the CPU? Not maxing out the CPU may actually be a good sign. Because you are being vague on the details, I am free to interpret your data like this:
- C# had to serve less requests because it maxed out the CPU
- Elixir was able to serve more requests and have spare CPUs
If that’s the case, I will pick the second, thank you.
That’s why when talking about benchmarks, we need numbers and methodology. There are about hundreds of things that could go wrong. Or even when the measurement is right, we take the wrong conclusions. So unless you can provide applications, benchmark tools and methodology, there is nothing to conclude and nothing to discuss.
Can you please provide an actual example? Please let us know your OTP version and OS too. In literally years benchmarking Elixir applications with tools like wrk, I did never bring it down. Even when opening 2 million connections - where we used 40 different client machines to benchmark a single server.
It is not about bad requests bringing down the server but how you semantically react to those. I have literally seen frameworks and libraries rescuing OutOfMemoryError and putting systems in an unmanageable state because of that.
Still, focusing on bad requests is a gross misrepresentation of what fault tolerance means on Elixir. It is also drastically undervalues the benefits of process in designing those systems. Some examples:
- Ecto being fault-tolerant means safer design around connection pools (and leaking of connections)
- Phoenix being fault-tolerant means we can easily multiplex multiple channels over the same websocket connection and save on system resources, while also scheduling on CPU and IO bound work
- Ecto being built on top of processes grants an excellent amount of visibility into the system. You can navigate process trees, inspect the pool memory, state, queue size and more, etc
- Phoenix being built on top of processes means no stop the world GC and per process garbage collection
And the list could go on and on.
You did not. You made vague statements. “It isn’t fast”. For what? Compared to what? You said it “doesn’t maximise the CPU” but you didn’t provide an example workload. It fails during benchmarking? How? What errors? Under which scenarios?
Yet we see companies using it for data processing with GenStage and Flow. Or for the web with Phoenix. Or for embedded devices with Nerves. I recommend folks to look at videos from conferences such as Empex, ElixirConf, CodeBEAM and others to learn more about the variety of use cases BEAM is deployed to.
27
jakemorrison
I wrote this blog post about avoiding GenServer bottlenecks. The best architecture is generally to model the natural concurrency of your system, and Elixir makes it really easy and safe to handle concurrency. The language is also a pleasure to write in.
8
dimitarvp
I am actually still finding myself nodding in agreement with the title and your general premise – including the newer blog post.
I came for OTP. I stayed for the functional programming.
Additionally, Ecto and Phoenix already make very good use of OTP. So truthfully, if you use either (or both) you already are reaping the benefits of OTP, as stated by @cmkarlsson and others here.
4
nathanl
having just Phoenix and Ecto be fault tolerant buys you nothing. Any modern web framework (regardless of language) can handle bad requests without bringing down the server
Fault tolerance isn’t just “what happens if there’s an exception”, it’s also how you handle load. If you can’t accept connections or if you take 30 seconds to reply, you’re effectively down.
What follows is my theoretical understanding.
Many web frameworks require running 1 OS thread per web request. Eg, you explicitly configure how many processes and threads per process to run if you’ve got a Rails app running on Puma. If you have 16 total threads and you get 17 simultaneous web requests, one of them is waiting in line. The 16th user is (hopefully) getting a nice response, and the 17th is hung entirely. Anyone making 20 requests at a time has got you with a denial of service attack.
Phoenix (via Cowboy) runs one BEAM process per request, and we know we can run millions of those. A million simultaneous web requests will probably hit some other bottleneck, like the database, of course, but Elixir itself will just keep adding processes as needed, each one slowing down the existing ones very slightly as they share scheduler time, giving a very smooth degradation under load, all without you having to think about it.
5
AVee
Web frameworks generally are have been moving away from that for quite a while now. Basically since the C10K problem (a term coined in 1999) and the C10M problem became a thing. This has been sped up recently due to better OS support (epoll, IOCP) and even more so because of better language support (async/await in C# for example).
Also, the 17th request can be handled by just starting a new thread (or OS process) in even the most primitive webserver. That may not be the most resource efficient way to do it, but it certainly isn’t a denial of service. How do you think the current internet could even function of that was true?
See How MigratoryData solved the C10M problem: 10 Million Concurrent Connections on a Single Commodity Server – Scalable Realtime Messaging for an example of doing 10 million concurrent connections with java from 2015. And note how the article is more about OS and network config and not at all about how to write the code, because that is a solved problem.
1
jeremyjh
He’s not wrong. His example of Puma - 16 threads - is the default for that Webserver: GitHub - puma/puma: A Ruby/Rack web server built for parallelism · GitHub
You can set it to 16 or 16,000, either way when you run out of threads the new requests are blocking.
1
dimitarvp
OS threads are a limited resource and most OS-es struggle under pressure when having to spawn thousands of them.
This is as DoS as it gets. Make a lot of requests that are costly for the OS and the hardware and they buckle under pressure. Maybe we have different definitions of DoS in mind?
Current internet is barely limping as is evident by the average load times of most websites and the fact that the best way to kill a website is to post a link to it on a popular aggregator like HackerNews or Reddit.
Apart from the few smart folks that setup a main website hosted on services like Netlify / CloudFlare, most websites are a DoS waiting to happen.
It’s true that a good chunk of a well-oiled webserver magic lies in OS configuration, everybody knows that.
But to claim that how to write the code for it is a solved problem is simply false.
0