Cowboy-Example is slow

Questions / Help
1

Hi Guys,

I have found the Cowboy-Elixir-Example and only get about 11.52 [#/sec] under an “Hello World” Benchmark.

I have used for my other projects the Cauldron HTTP/SPDY Server and provided out of the box 4-6k [#/sec] … so do i miss something in this Example or why is cowboy the super fast ErlangHTTP Server so slow?

Thank you for your help :slight_smile:

2 Likes
2

Even if I can’t confirm the 10 Req/Sec, I do have not much myself…

(Stripping banner and progress from abs output)

$ ab -kc 250 -n 5000 http://localhost:8080/static/static_example.html

[…stripped…]

Server Software:        
Server Hostname:        localhost
Server Port:            8080

Document Path:          /static/static_example.html
Document Length:        1186 bytes

Concurrency Level:      250
Time taken for tests:   100.223 seconds
Complete requests:      5000
Failed requests:        0
Keep-Alive requests:    0
Total transferred:      6855000 bytes
HTML transferred:       5930000 bytes
Requests per second:    49.89 [#/sec] (mean)
Time per request:       5011.144 [ms] (mean)
Time per request:       20.045 [ms] (mean, across all concurrent requests)
Transfer rate:          66.79 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   1.1      0       7
Processing:  5000 5008   9.3   5004    5047
Waiting:        0    6   8.2      3      41
Total:       5001 5008  10.1   5004    5052

Percentage of the requests served within a certain time (ms)
  50%   5004
  66%   5006
  75%   5009
  80%   5011
  90%   5025
  95%   5032
  98%   5043
  99%   5049
 100%   5052 (longest request)

I did some experiments using different levels of concurrency, and I did reach better R/sec values when doing more concurrent requests, but also much higher droprates because I ran out of file handles, something the @Phoenix-Core-Team can tell you more about :wink:

Also I tried running on dev and prod environment, which didn’t changed much, RPS were pretty much the same (difference was second digit after the comma).

And phoenix does proof that cowboy can be faster, the following was done using a clean phoenix app as it gets generated by mix phoenix.new:

$ ab -kc 250 -n 5000 http://localhost:4000/

[…stripped…]

Server Software:        
Server Hostname:        localhost
Server Port:            4000

Document Path:          /
Document Length:        1949 bytes

Concurrency Level:      250
Time taken for tests:   1.663 seconds
Complete requests:      5000
Failed requests:        0
Keep-Alive requests:    5000
Total transferred:      11510000 bytes
HTML transferred:       9745000 bytes
Requests per second:    3006.22 [#/sec] (mean)
Time per request:       83.161 [ms] (mean)
Time per request:       0.333 [ms] (mean, across all concurrent requests)
Transfer rate:          6758.13 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   1.1      0       8
Processing:    21   81  18.0     81     124
Waiting:       21   81  18.0     81     124
Total:         21   82  18.5     81     131

Percentage of the requests served within a certain time (ms)
  50%     81
  66%     88
  75%     93
  80%     96
  90%    107
  95%    114
  98%    122
  99%    126
 100%    131 (longest request)

So I do think, that in your example linked above something is configured in a bad way and phoenix does do a better job configuring cowboy :wink: I am sure, putting some effort into this issue the linked example can get into the same area of responsiveness.

But I have to admit, I can’t give any numbers for cauldron on my machine right now.

2 Likes
3

I’m not sure this is the reason, but I know in the past ab had some issues
with cowboy and phoenix and reported strange results. Can you try with wrk?

1 Like
4

i will give it a try - but ab did a good job with Cauldron and Phoenix on my machine …

1 Like
5

Jesus!

Martins-MacBook-Pro:Downloads bMalum$ wrk -c 100 "http://127.0.0.1:8080"
Running 10s test @ http://127.0.0.1:8080
  2 threads and 100 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    15.26ms    1.24ms  42.81ms   90.28%
    Req/Sec     3.29k   167.42     3.62k    84.16%
  66095 requests in 10.10s, 14.18MB read
Requests/sec:   6542.07
Transfer/sec:      1.40MB
Martins-MacBook-Pro:Downloads bMalum$
1 Like
6

That’s still really slow. Couple year old macbook pro:

$ wrk -c 400 -t 4 http://localhost:8081/
Running 10s test @ http://localhost:8081/
  4 threads and 400 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    47.55ms   96.54ms 618.75ms   88.08%
    Req/Sec    12.07k     1.87k   16.16k    71.50%
  480325 requests in 10.01s, 47.72MB read
  Socket errors: connect 0, read 280, write 0, timeout 0
Requests/sec:  47984.60
Transfer/sec:      4.77MB

Code: https://github.com/benwilson512/cowboy-example

2 Likes