Hi all, just wanted to say thank you for all of your timely help on this. I have been working through the suggestions a bit at a time as time permits, and I will put up some code as soon as I can. So far, turning off all of the output to stdout and building an exrm release have provided some improvements - though there are still a few scenarios where MIX_ENV=PROD outperforms even the exrm release.
I’ll update as soon as I have some more information. Thanks again!
Any updates, Matt? I came to this thread from Sasa’s blog post and I remember a thread on Hacker News performing benchmarks at Rackspace. In your original post you had mentioned going back to Java, so I thought I’d link you to their tests: Phoenix Showdown Comparative Benchmarks @ Rackspace · GitHub
Few things to consider
If you can send the sample code, I can take a look and suggest you better.
Thanks for this @sudostack - I have been concentrating on getting some benchmarks for Go, Java and Elixir since last I checked in. Elixir is next up and I hope to get some more data this week. Thanks for the link to the tests - would be awesome to see some updated numbers. It’s impressive to see the throughput that Phoenix was getting back then. I’ve been getting <20k RPS on my macbook.[quote=“subbu05, post:15, topic:832”]
If you running tests from outside AWS you might see some delay because of connecting time to AWS.2.Try to have AWS instance with SSD volumes instead EBS volumes.
[/quote]
I’m going to try hitting it from either the same box or one within the same VPC. Good advice on #2. We were spinning these up on Docker instances, which people don’t seem to do with Elixir. I haven’t seen a good reason yet, but I am curious as to what overhead is introduced by Docker.
I have been doing similar benchmarks etc.. for my application which is a json api - and I have found that elixir / phoenix is not the fastest thing out there (nor does it claim to be), but combined with the balance of productivity vs performance in my opinion it beats scala / java / go and the rest, but I am considering it coming from ruby / rails so its similarity to ruby was important too. Saying that, I have worked with scala and java before and cannot see the frameworks such as play being anything like as easy to develop with, but they probably will get more requests per second out of a single box, but elixir scales in a more predictable way is what I have read when adding more hardware - not got to that point yet though.
Phoenix is fast, but only as fast as your pipeline allows it to be. The point of phoenix is to be ‘near’ the fastest while having beyond intense reliability.
However my main point for this post, *DO*NOT*TEST*FROM*WINDOWS*. We learned that the hard way from work. If a server is on windows or the testing client is on Windows then (at least Windows 10) introduces a near 200ms latency on initial connections while it ‘fills the TCP buffer’, and at least on windows 10 we’ve tried everything to disable it, registry edits, setting the TCP connection with NONAGLE and a hundred things in-between…
I know this has been looong ago, but I have to clarify this for myself: how is it possible that Phoenix, which is built on top of Plug, has a (slightly) lower latency and considerably better consistency (lower σ)?
Given how old these benchmarks are it’s anyone’s guess, I’d suggest they be redone before anyone worries about analyzing their results.