Tough to tell. A few observations:
-
BLUF: Of course this is an apples and oranges comparison. I personally couldn’t give two hoots how well we perform against other languages (especially natively compiled ones running on base libraries); that’s not a game I have any interest in playing or one that has any winners. It’s MONGODB IS WEB SCALE all over again and I have better things to do with my time than to engage in the comparative aspects of this. The two contestants aren’t even playing the same sport
-
The PR’s setup looks fine (it’s not a matter of app configuration). I didn’t look at any of the lower level OS / BEAM tuning details
-
He’s using
m7a.largeinstances, which at first glance look like they’d perform a smidge worse than the instances we use for microbenchmarks in CI. From that perspective the results seem roughly correspondent with what I’d expect in absolute terms. -
I’m a little worried by the growth numbers that Bandit demonstrates. CPU usage shouldn’t be growing without bound like that, and (as he states) that’s likely the root cause of the lacklustre numbers elsewhere
-
There’s not really a whole lot of actionable steps to take based on this data. We really do need a better benchmarking environment (ideally one that runs as part of CI), as the microbenchmarking setup we use now just doesn’t get to the absolute scale needed to reproduce these sorts of situations ‘in the lab’, which is a necessary precondition to be able to improve them in Bandit. If anyone is looking for a place to help, that’s probably the highest value way to do so
One thing I noticed though is that throws and exits emit 
