Performance - Best practices?

merlin

Hi all,

I’ve run the Techempower fortune benchmark locally, Elixir/Phoenix and Golang/fasthttp.
Removed all unnecessary code in both Elixir and Go.
I’m trying to get the best performance out of the Elixir/Phoenix version.

Tuning for Elixir/Phoenix:

disabled all unnecessary plugs in router.ex:

pipeline :browser do
plug :accepts, [“html”]
end
set DB adapter pool size to 40
used the same HTML template as Go
set logging to error
compiled protocols
start server: MIX_ENV=prod PORT=4000 elixir -S mix phx.server

No tuning for Go.

I use wrk to perform the load tests:
wrk -t6 -c12000 -d60s -T10s http://.../fortune

Client is physical and server is a domU/2vcpu/4GB. OS: FreeBSD. 10 runs for each load test.

For Elixir/Phoenix:

Average latency 408ms, max 8s
Req/s 2400

For Golang:

Average latency 123ms, max 1s
Req/s 8600

I did my first Go tests with net/http and peaked at 4000 req/s so I replaced it with fasthttp and got 8500+ req/s
Are there faster alternatives to Cowboy ?
How do you fine tune production environment ? What OS is best suited for the Erlang VM ? What FS ? specific OS tunables ?

Thanks

16 comments

/phoenix

2 10099 16

2017-12-14 16:42:08 UTC

Showing Posts 1 to 10

Show Best Posts
Show All (oldest first)
Show All (newest first)

OvermindDL1

Uh, you have something MAJOR wrong there. Can you upload this to github and give us commands to git clone ... it then a single (set of?) commands to run our own tests? Even the golang average latency of 123ms seems crazy high!

Post #1

outlog

-c12000 isn’t that 12k keepalive connections, seems a bit much..

techempower is not real world - quite the opposite actually.. so do your own benchmark similar to what your requirements are..

wrk is not a great benchmark tool.. use something like https://gatling.io

also allow me to suggest this video

Post #2

outlog

also since it’s go vs elixir(otp/erlang) this one is great at understanding latency and GC handling etc.

video:

Post #3

idi527

Are there faster alternatives to Cowboy ?

There is elli TechEmpower Benchmarks Round 14

 +        for _ <- 1..n do
 +          id = rand_id()
 +          [row] = Hello.SQL.query(conn, "world_by_id", @world_by_id, [id])
 +          world(row)
 +        end

Might be replaced with a map, since for comprehension in elixir does a reduce where a map would suffice, IRRC. Though it’ll hardly change anything.

I’m not sure if .eex templates compile to iolists. If not, then that’d also be a possible improvement in performance.

Post #4

OvermindDL1

They do.

Post #5

merlin

Thanks for Elli, I’ll give it a try

To reproduce this test: just co the techempower code, elixir/phoenix and golang/fasthttp versions.
For the phoenix version you may also bootstrap an empty app and add a fortune MVC.
Then add some basic tuning as I’ve done. The average latency is high probably because of the number of incoming connections. That’s the point. Same conditions for the Go version. I’d like to optimize at least:

the average latency ratio between Elixir and Go: almost 4
get rid of the max latency of 8s, which is a lot

The techempower/fortune benchmark gives us a good basis to apply the most basic tuning. That’s what I’m interested in now, and that’s why I use wrk and not JMeter. Later, if we choose Elixir, I’ll perform more advanced load tests.

Except plugs, DB pool size, logging are there other config. parameters I could change to improve performance ?Beam parameters I could tune ? JVM-style ?

Post #6

idi527

Then at what point and why do they get turned into binaries? https://github.com/idi-ot/eex_test

iex(1)> EExTest.fortune_html [a: "a", b: "b"]
"<!DOCTYPE html>\n<html>\n  <head>\n    <title>Fortunes</title>\n  </head>\n  <body>\n    <table>\n      <tr><th>id</th><th>message</th></tr>\n      \n        <tr><td>a</td><td>a</td></tr>\n      \n        <tr><td>b</td><td>b</td></tr>\n      \n    </table>\n  </body>\n</html>\n"

I would’ve expected something like this

iex(3)> EExTest.custom_fortune_html [a: "a", b: "b"]
["<!DOCTYPE html>\n<html>\n  <head>\n    <title>Fortunes</title>\n  </head>\n  <body>\n    <table>\n      <tr><th>id</th><th>message</th></tr>\n",
 ["<tr><td>", "a", "</td><td>", "a", "</td></tr>", "<tr><td>", "b", "</td><td>",
  "b", "</td></tr>"], "    </table>\n  </body>\n</html>\n"]

Some tests

# render small list

iex(6)> :timer.tc(fn -> Enum.each(1..100_000, fn _ -> EExTest.fortune_html([a: "a", b: "b"]) end) end)
{480068, :ok}

iex(7)> :timer.tc(fn -> Enum.each(1..100_000, fn _ -> EExTest.custom_fortune_html([a: "a", b: "b"]) end) end)
{279135, :ok}

# render empty list

iex(10)> :timer.tc(fn -> Enum.each(1..100_000, fn _ -> EExTest.fortune_html([]) end) end)
{144888, :ok}

iex(11)> :timer.tc(fn -> Enum.each(1..100_000, fn _ -> EExTest.custom_fortune_html([]) end) end)
{74700, :ok}

# render bigger list

iex(12)> data = Enum.map(1..1000, fn i -> {i, i} end)

iex(14)> :timer.tc(fn -> Enum.each(1..1_000, fn _ -> EExTest.custom_fortune_html(data) end) end)
{277821, :ok}

iex(15)> :timer.tc(fn -> Enum.each(1..1_000, fn _ -> EExTest.fortune_html(data) end) end)
{1078125, :ok}

Post #7

idi527

You might want to read this series of blog posts (this one is the last one, I think) http://dbeck.github.io/Wrapping-up-my-Elixir-TCP-experiments/. There is a part about beam parameters near the end and in the comment section. And docs (Emulator Flags section) erl — OTP 29.0.2 (erts 17.0.2)

Post #8

merlin

Thanks

This looks really interesting esp. the FreeBSD part

Edit: apparently Mr Beck switched to Rust a year ago. Looking for a C++ alternative ?
It’s getting harder and harder to choose the right language: Java, Scala, PHP, Go, Erlang, Elixir, Rust, Ruby, Crystal, Pony, Nim … Oh did I forget JS ?