Load Testing a Phoenix app using siege

fireproofsocks

Does anyone have resources/books/posts that they can recommend for how to tune a Phoenix app? I have a Phoenix app running as part of an umbrella deployed to Gigalixir and I want to put it through its paces. I have downloaded Siege and wrk and I have tried hitting both my local and my Gigalixir deployment. I’m not wowed by any of the numbers I’m looking at… for comparison I’ve tried hitting some of the other PHP apps we have running, and the stats are in the same ballpark… in some cases, the error coming up with Phoenix are a lot higher (!). WRK reported more favorably, with Phoenix performing approximately 10x more connections handled per second.

I’ve started kicking the tires on the observer, and I was able to bring down the number of database connections I used, but really I’m just groping in the dark. Can anyone point to a tutorial or something that relevant?

Thanks!

6 comments

/phoenix #testing #benchmark #gigalixir #tuning

1 3044 6

2019-12-13 18:08:33 UTC

Showing Posts 1 to 6

Show Best Posts
Show All (oldest first)
Show All (newest first)

idi527

Are you running wrk and siege from your local machine when testing the app on gigalixir? That wouldn’t probably show any useful data, since the network between your local machine and the server is not under your control.

So you’d need to run wrk and siege from within gigalixir, preferably the same AZ as your app.

Maybe The Road to 2 Million Websocket Connections in Phoenix - Phoenix Blog would help.

Post #1

benwilson512

Author of Craft GraphQL APIs in Elixir with Absinthe

In addition to @idi527’s great comments, it would be helpful if you supplied some information. How much CPU / RAM do your servers have, what parameters are you passing to Siege / Wrk?

Post #2

fireproofsocks

siege -d3 -c100 https://my.app/
wrk -t12 -c400 -d30s https://my.app/

The error that comes back most frequently from siege is

[error] socket: unable to connect sock.c:249: Operation already in progress

Sample response from WRK:

  12 threads and 400 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    91.13ms   19.46ms 343.70ms   76.15%
    Req/Sec   360.04     49.02   595.00     76.08%
  128189 requests in 30.10s, 58.92MB read
Requests/sec:   4258.29
Transfer/sec:      1.96MB

Sample siege output:

Transactions:		         761 hits
Availability:		      100.00 %
Elapsed time:		       14.66 secs
Data transferred:	        0.01 MB
Response time:		        0.50 secs
Transaction rate:	       51.91 trans/sec
Throughput:		        0.00 MB/sec
Concurrency:		       25.74
Successful transactions:         761
Failed transactions:	           0
Longest transaction:	        2.57
Shortest transaction:	        0.15

The Gigalixir instance has 0.5 GB Memory and 1 Replica. I’m not sure what that has for CPU.

Post #3

benwilson512

Author of Craft GraphQL APIs in Elixir with Absinthe

I’m not sure what this means but this sounds more like a siege issue? 4k req/ sec from wrk on what is probably 1 CPU is really rather good particularly given that you’re also doing it over the general web.

Post #4

cmkarlsson

I’d recommend that you have a look at wrk2 (GitHub - giltene/wrk2: A constant throughput, correct latency recording variant of wrk · GitHub) as well. It is especially written to consider something called coordinated omission (explained in the readme for wrk2 and there are some good talks on it as well if you search for it).

It also allows you to set the rate (perhaps wrk can do this today as well) of requests so that you don’t overload your system (unless that is what you are testing).

For more elaborate testing you have

tsung (http://tsung.erlang-projects.org/). It feels a bit “old” but it pretty good. I think this is what was used when they tested the 2M web socket connections.
gatling (https://gatling.io/) I found quite pleasant to work with.

Post #5