steffend

steffend

Phoenix Core Team

Plug.Conn.read_body vs. Plug.Conn.chunk performance

Hello there,

I’m currently working on implementing a WebDAV server using Plug/Cowboy, inspired by the great Python wsgidav library. So far read only access works fine and when downloading files using Plug.Conn.chunk I’m seeing very good performance (1,88GB/s - GigaBYTES from local SSD to /dev/null) with less than 50% scheduler utilization:


But while implementing initial write support I’ve noticed that the other direction, using Plug.Conn.read_body maxes out at about 200MB/s - even when just discarding the output:

You can try this yourselves by cloning the repository (write_support branch) and running the demo project from the demo folder. Then starting the server is as easy as using iex -S mix after the usual mix deps.get step.
For uploading, you can use curl -X PUT -F 'data=@/dev/random' http://demo:demo@127.0.0.1:5001/tmp/test.null.
For downloading, first create/download a sufficiently large file and then use wget -O /dev/null "http://127.0.0.1:5001/tmp/10G" --user demo --password demo (replace /tmp/10G with the file path).

I tried tweaking the Plug.Conn.read_body opts, but did not find anything that improved this.

Maybe someone with deeper insight into plug/cowboy has an idea why the Plug.Conn.read_body direction performs so bad. I’d expect both directions to have roughly the same performance and scheduler usage.

Thanks for any hints!

First Post!

Exadra37

Exadra37

I am not an expert in this area, but in my opinion isn’t the hint for the reply in the name of the functions?

The read body function may need to wait for the whole file to came in to process it, while the chunck function will process it in chunks?

Most Liked

steffend

steffend

Phoenix Core Team

That’s not the case. As documented, read_body reads the body up to a given length. So both are processing the data in chunks. Otherwise the memory usage would explode when reading indefinitely from /dev/random.

Where Next?

Popular in Questions Top

jononomo
I am trying to figure out how Mix knows whether the environment is test, dev, or prod – where is this set? Thanks.
New
hariharasudhan94
lets say i have a sample like a = 20; b = 10; if (a > b) do {:ok, "a"} end if (a < b) do {:ok, b} end if (a == b) do {:ok, "equa...
New
Brian
What is the proper way to load a module from a file in to IEX? In the python world, doing something like this pretty standard: from ....
New
ashish173
I am using Ecto timestamps with postgres, I can see the timestamps() use the :naive_dateime but for my use case I wanted to store the ti...
New
PeterCarter
There are pre-rolled solutions for other frameworks that do work. However, Phoenix does not seem to have these. Have people had good expe...
New
SoCreat
i’m a new one to elixir which editor can i use vs code? or atom? Thanks! :smiley:
New
JorisKok
I have a server on AWS, and was running a load test using artillery. When looking at the Phoenix dashboard I see the Ports going to 100% ...
New

Other popular topics Top

baxterw3b
Hi guys, i’m new in the Elixir world, and i have to say, that i love it! i’m having some problem to understand anonymous functions with ...
New
Qqwy
Original source of discussion: This topic on the Pragmatic Programmers’ Functional Web Development with Elixir, OTP, and Phoenix forum. ...
New
vonH
When I run the Plug and I recompile I wind up having to use Ctrl C to quit iex and start again. Witht the help of rlwrap I can use the cu...
New
Brian
What is the proper way to load a module from a file in to IEX? In the python world, doing something like this pretty standard: from ....
New
nsuchy
Hi. I’ve noticed that Windows Powershell has it’s own IEX command and you cannot access Elixir’s IEX due to the conflict. This isn’t a cr...
New
sen
Hi All, I set a environment variables in dev.exs , like below code. when i start server, how can i set the ${enable} value? thanks. d...
New

We're in Beta

About us Mission Statement