Debatable. Go is proven to have one of the best startup times of most modern statically typed compiled languages (0.5 - 2.0ms on average). Rust I think was slightly behind (3.5ms) with OCaml a bit after it – at 4.0 - 8.0ms startup time. (Can’t find the link to that quick “study” which was basically hello world programs in many languages)
If we are talking about pure socket performance I think Go is quite adequate. Goroutines make this a breeze. Now the language has various problems that we agree are nasty (like the untyped data escape hatch) but in this department it’s good and the stdlib is quite impressive (especially in I/O; I was able to just redirect traffic from customer HTTP request directly to S3 with a Go middleware service I wrote – haven’t achieved the same in another language yet, do you know if Rust and OCaml would allow me to do the same?).
Since microservices pops up every now and then, I’d think Go, Rust and OCaml are among the top candidates for that. But for hundreds of thousands of requests per sec… yeah, not sure.
The issue is not just socket performance but also how well the system handles load. Unfortunetely when you are doing work in Go, Go creates garbage and the GC eventually has to run. The Go GC is tuned to minimize pauses at the expensive of extra memory growth, and with literally 100k new connections per second that is going to be a LOT of memory churning unless you are exceptionally careful to use pools everywhere, and I think it’s socket structure allocates regardless so that’s hard to prevent. OCaml has it even worse. Compare to in Rust/C++/etc… where you have significantly finer control over memory, you can get to the point where the only allocations happening are the socket kernel structure itself by good use of pools and not using heap allocations, depending on what work is actually being done on the socket of course. Go is perfectly sufficient for efficiency as long as its GC doesn’t hit any of those worst cases (excepting Go’s other issues like it’s verbosity, difficulty to read, void passing way too much stuff, etc… etc…).
OCaml is like Python, it has a GIL, so you’d need to multiprocess across the listen socket, which will both reduce efficiency of the listening socket itself, and OCaml is heavily built on a GC, although it is a very tuned GC that takes advantage of a lot of things about the language to be faster than most, it is still a GC with a lot of even the assembly coded data being boxed up to make things even worse. ^.^;
So the problem is the GIL? I have the impression that these languages still perform very well when they are I/O bound since raw processing power isn’t needed.
Generally so except with 100k new connections per second is a bit… extreme… You are pushing up against the limits of the kernel syscall interface at that point… ^.^;
True, but that’s no longer a weakness in the language.
So since I have you here, what would you recommend OCaml for? I’m taking some baby steps learning it and even though I’m unhappy with the fragmented tooling, the language is mighty fine so far.
Honestly my main uses of it are as a type-safe and faster version of Python. So what I would normally script up in Python I generally use OCaml for unless Python has a very specific library that it doesn’t have (OCaml has a lot of libraries though, but not near as much as python).
What about the tooling is fragmented? It was like 10 years ago, really badly, but nowadays there is the official Dune build system, the official Opam library repository, etc… etc… It seems really solid to me now but then again I remember the ‘bad days’ ^.^;
Can’t exactly put my finger on it. Scanning through OCaml’s discourse once a week or so, I don’t see the community agreeing even on elementary things like source formatting for example. Also Dune is… weird. LISP syntax in your build system. Kind of strange. But I guess it’s an acquired taste?
I have no concrete technical complaints so far. But things don’t seem so streamlined and widely agreed on as in the Elixir ecosystem somehow.
The source formatting bit is mostly because refmt absolutely murders any semblence of formatting in OCaml code, it’s pretty horrible, what is mostly used is just ocp-indent, which keeps your formatting but re-indents to keep things correct. ^.^
Dune is just a rebranded jbuilder, which was the most used one for years before hand, so it’s quite well known and quite powerful and quite fast. LISP syntax’s are pretty common in a lot of build systems over the decades I’ve noticed…
Elixir is also brand new, this would be more akin to comparing it to Erlang. Give Elixir time and there will be a host of things out too, like there is for Ruby or Python.
In any case though, back to original topic – I wouldn’t blame any language if the runtime hits kernel’s limits. In that regard almost any language is good if you’re gonna saturate your VM / bare metal in terms of I/O, wouldn’t you agree?
OCaml started appealing to me lately due to its typing system and ability to slap you on the wrists if you make a mistake – however that doesn’t come for free, you have to get better to reap the benefits (like use polymorphic variants instead of plain ones?).
At least on linux there are a variety of ways to handle network I/O, including some methods for minimizing the kernel state transitions. For those you either have to hope a language has a binding to those really low level methods, or you need to write it yourself in C or whatever the language binds to, in addition to marshalling the data into the language runtime if it’s not a low level language like C/C++/Rust/etc… ^.^;
In general normal variants are better than polymorphic variants. Polymorphic variants were designed for the specific use-case of passing ‘unknown’ information through a system, like via plugins, or so. As an example, exceptions in the language are actually an open polymorphic variant (a type of polymorphic variant that is kind of ‘scoped’ to a specific namespace, in this case I think its called exc or something like that). ^.^
Ah, true. I keep forgetting about that. Do you have any info which languages are ahead in this regard? I heard Rust is making good efforts – but people also praised Golang for using certain syscalls that help as well. But I admit I haven’t done an exhaustive research. How about you? Do you also know how does OCaml fare?
C++, C, Rust, Go, Nim, and Nodejs I know are ones with efficient stdlib/libraries for this, although on both Go and Nodejs they have to marshall things to their own internal format, which adds needless overhead. Nim I don’t think has to marshall it but it does have a simple GC for reference cycle catching and so forth so it’s “usually” not as much of a hit as GC’s in, say, Go or Nodejs.
I don’t actually know about those calls for OCaml, but as OCaml can pretty trivially bind to C libraries from within the language itself (a great FFI system) then OCaml can consume any existing C library with ease so you could always have it use something like libevent or libmill/libdill (whichever one was function based) or whatever else.
Yeah but how about no. Extra complexity. I’d like productivity and less WTFs/minute better.
Many things can be done “trivially” but these are famous last words in many areas, not only programming. Let’s go for stuff that makes the right calls out of the box.
Also, OCaml has a pretty nice and strict compiler, why should we even bind to C libraries? How much assurance do we have that they don’t have an obscure buffer overflow/underflow bug, or a subtle zero-day?
I understand the appeal but IMO when I pick a language I am looking for everything that I need in it to be written in the said language, otherwise can you even claim to reap the benefits of its compiler and runtime? If I wanted to assemble Frankenstein monstrosities I could do that even 10 years ago with Java and C/C++ libs and bash. Not looking to go back to these days.
That is a big reason why Rust is basically rewriting the entire C/C++ library ecosystem at this point! ^.^
Rust is a lot more noisy in syntax, and significantly slower to compile than OCaml, but if I’m going for any bigger project like that listed in this thread then I’d absolutely go for it now (or C++, but honestly I’d try to push for Rust since I’d learn more, unlike C++ where I’d be able to get it done more quickly but I’d have to trust the libraries more, which I decently do of the ones that I pick anymore, Rust is more… trustable in general).
And I strongly applaud these efforts. I am looking forward to use stuff like find, xargs, grep, awk and more rewritten in Rust! It’s time to leave C for microcontrollers only and C++ probably for mission-critical NASA-level software (or PC games).
I’d argue Go and OCaml are also quite good for that as well and I don’t really subscribe to the idea that GC == bad thing. Go has sub-millisecond GC pauses and even if it doesn’t collect everything at one step it still uses less memory than a naive C tool that reads its entire input in memory. I imagine the same is true for OCaml as well.
This is my main gripe with Rust. Hardcore programmers as yourself don’t mind quirky syntaxes but the practices that a Rust project needs even for 5-people teams simply don’t scale. People make mistakes, skip linters, forget to run code formatters, skip GIT pre-commit hooks when in a hurry to fix a critical bug, turn off the --warnings-as-errors compiler flag when it’s Friday evening, etc. You simply cannot rely on humans to always obey the rules.
Hence the language and its tooling has to hit hard. Rust and OCaml seem to do that but OCaml is much more succinct (although to be completely fair, some OCaml codes looks very cryptic).
I am rather surprised of your more cautious stance about OCaml, frankly. You advocated for it really often on this forum, many times.
And nah, Rust is great on microcontrollers and critical software too.
It says sub-millisecond because it’s tuned to be very bad at reclaiming, which can hit you really really hard, especially when working with a lot of small things, like connections. Not only does it mean that memory grows a lot faster (unbounded until it is forced to either do a larger collection or die, like via the linux OOM manager), but also means it all gets scattered across memory, hurting access times on things like just accessing non-local data structures. And yes, OCaml has the same issue, most GC languages do (I’ve seen Java die hard in similar situations at my last job).
Rusts’s compiler is awesome, slow but quite comprehensive in what it catches with pretty great error messages! OCaml looks SML’y, which coming from a ruby/javascript/C world may look cryptic but it is entirely unambiguous, clear, succinct, and fast.
Use the right tool for the right job. I love the OCaml language, compiler, and ecosystem, but I would not use it for a task like this thread proposes, I wouldn’t even use the BEAM for what this thread proposes if using a single machine, the BEAM I’d use if using many machines and sharding out the connections, but I wouldn’t on a single machine. Just opening and closing an HTML connection as fast as I can in a quick C++ program using libevent with minimal header and path parsing and returning a static packet of data with an empty body and minimal headers (just 2) using keepalive to an extreme with 5 dedicated threads each having 20 connections (about the highest throughput I was able to hit was with this count) to saturate the cores on the machine, and I can almost hit 200k connections per second (on this fairly old but still powerful 6-core machine) sustained, almost all of that cost is the kernel socket setup and communication overhead, and that’s with doing nothing else, no database access, no heap allocations (no heap access at all actually), everything is in active memory and resides in the CPU cache. A simple Elixir plug/cowboy server with the same setup and all was able to hit 56k connections per second, and the BEAM also uses very well optimized socket I/O (especially with cowboy). With a simple ruby echo server I cloned here I was able to get ~3663 connections per second, and with a simple rails server I just cloned that just returns a text body (both of which were hosted via puma or whatever it was, it looks like it uses multiple processes) I was able to hit 504 connections per second. Grabbing a couple of Rust web servers and setting up trivial empty body endpoints in them and I get 126k with iron, 97k with nickel, and 87k with rocket, and these are some trivial servers, and I know that actix and some other rust servers are faster but requires a touch more then a few lines in a single file, although I am curious about actix (it’s very OTP’ish and thus built for larger systems, I wonder how it will do on a trivial web server test since it has some Actor overhead), setting it up… and it got 208.5k/second (oooo how nice… I really need to learn this more, faster than my simple libevent web server).
For this problem in this thread if I’m constrained to a single system I’d absolutely use Rust and try to figure out a way to make the remote connections stay connected to save on the TCP setup each time (or switch to UDP without html or so) all depending on ‘what’ information is being sent and how reliable it needs to be and all.
EDIT:
Indeed! Rust really can overthrow C/C++ in everything C/C++ can do (and it compiles faster in debug then in the highly optimized releases I’ve been having it do for this post).
The only thing I’d say here (because I agree with everything else!) is that I hate the fact that we’re gonna replace one imperative language with a FP/OOP language with a very quirky syntax and a rather fragmented community.
I kind of hoped for something succinct like OCaml or LISP…
