Speed difference between chunk http response and non-chuncked

Questions / Help
,
1

Hello everyone I’m studying about http streams and I notice a big difference in performance when returning everything at once and when streaming and that made me think if streaming a big file is really worth it or if there is something wrong with my implementation (most likely that haha) So if anyone could help me understand this situation I’d appreciate a lot. Thanks

non-stream implementation

image
image1200×481 40.3 KB

  get "/no-stream" do
    pokemons = File.read!("file.json")
    conn |> send_resp(200, pokemons)
  end

Stream implementation :

image
image1200×477 40.6 KB

  get "/" do

    conn = conn
    |> put_resp_content_type("text/event-stream")
    |> put_resp_header("Access-Control-Allow-Origin", "*")
    |> send_chunked(200)

    File.stream!("file.json")
    |> Jaxon.Stream.from_enumerable
    |> Jaxon.Stream.query([:root, :all])
    |> Stream.map(fn element ->
     {:ok, pokemon}  = Jason.encode(element)
      conn |> chunk(pokemon <> "\n")
    end )
    |> Stream.run

    conn
  end

As you can see the stream implementation has a latency MUCH higher even though the memory peak is a little bit lower. The question here is trying to understand if this is normal or if I messed up somehow I understand if is not possible to achieve the same latency as File.read! but the difference here is so high that made me wonder if something if off

BTW: the lib used to run the load test is called hey

2 Likes
2

Streaming is always slower than sending/processing the whole file at once. In this case I think the latency comes from the overhead of sending multiple http packets.

Usually you want to stream when the file can be very big, this will ensure that both the server will never run out of memory when loading the file in the memory and the client the same.

3

Looks like your streaming version does more work tbh, parsing and encoding JSON…

Also I believe send_file would be even faster :slight_smile:

1 Like
4

changing the endpoint to

 File.stream!("file.json")
    |> Stream.map(fn line ->
      conn |> chunk(line)
    end )
    |> Stream.run

made the endpoint 99% response time to 3s way higher than the 0.3s from file.read! but as you mentioned faster than the current json encondig…

5

When I think about when to stream I think of three cases:

  1. Time-to-first pixel is more important that absolute throughput. By streaming the user may be able to see something after the first chunk. In non-streaming the whole content has to arrive first. Very use case dependent.

  2. The content is being streamed from an external source (like S3) to Elixir and then transformed before sending to the HTTP client. In this case streaming may be more memory efficient and the stream-in/stream-out may manage the upstream latency and bandwidth better.

  3. The size of the content to be sent is non-trivial in terms of memory availability on the Elixir system. (as you’ve already pointed out).

I appreciate some of this may be more theoretical benefit rather than practical benefit so as always, testing and measuring is important.

5 Likes