azyzz228

azyzz228

Network optimization: 4x WS message size reduction when `push_event`

The slow network is known to be an Achilles heel of LiveView’s architecture.

Recently, I was working on creating a fast rendering map with 12,000+ points that are coming from DB. My server sent data in chunks of 4000 and it worked great on my localhost. But when deployed, network made it super slow.

I was able to reduce the WS message size with my data by 4 times (from 400kb to just 100kb) by using compression technique:

On the Elixir side,

  defp compress(data) do
    data
    |> Jason.encode!()
    |> :zlib.compress()
    |> Base.encode64()
  end

and on the JS side:

const decode_base64_decompress_to_json = (base64string) => {
    let base64_decoded = Base64.atob(base64string)
    let charCodes = [];
    for (let i = 0; i < base64_decoded.length; i++) {
        charCodes.push(base64_decoded.charCodeAt(i));
    }
    let inflatedData = pako.inflate(charCodes, { to: 'string' });

    const data = JSON.parse(inflatedData);
    return data;
}

I think this technique can drastically improve communication between the LiveView process and client-side JS when both are passing medium to large amounts of data. I found execution time of both functions + increased bundle size from JS dependencies (pako + Base64) to be negligible compared with the performance boost they generate.

I am happy to work on PR, though will probably need some support.

Most Liked

outlog

outlog

did you try websocket: [compress: true] ? - Phoenix.Endpoint — Phoenix v1.7.11

not sure what the default is.

tj0

tj0

I know this may sound silly, but it could be the way it’s being measured. I know firefox / chrome devtools measures the transferred and total bytes for http, but only do total bytes for websocket on my versions.

My websocket payload for a single request is approximately 90kB as measured by devtools. Measuring network traffic with btop on localhost, the total transfer was 467 kB after roughly 50 requests, so compression is definitely enabled for me.

Endpoint.ex

  socket "/live", Phoenix.LiveView.Socket,
    websocket: [ connect_info: [session: @session_options], log: false, compress: true]


mwhitworth

mwhitworth

Interesting idea! Maybe this and other properties of the LV data flow between client and server could be represented as a pipeline of plugs

When you talk about a “fast rendering map”, what is this exactly?

One point of view might be that there is a rough upper limit on the amount of elements visible on the screen at any one time (and likely to change), and so techniques that take into account the current client viewport somehow might be one way to cut down on the traffic

(I haven’t seen this done though!)

Where Next?

Popular in Proposals: Ideas Top

andreamancuso
Hey folks, This might sound niche, but I think it’s worth bringing up - especially given Phoenix’s reputation for being lightweight, por...
New
Jskalc
Hi everyone! Recently I was thinking a lot about the way HEEX renders lists. People are generally surprised about huge payloads being sen...
New
shahryarjb
When proposing or suggesting something please consider: As my experience implementing getBoundingClientRect as Phoenix.LiveView.JS funct...
New
pierrelegall
Problem Currently, List.first/2 and List.last/2 return a default value (or nil) when the list is empty. However, there are cases where an...
New
ffloyd
The Problem Currently, if I define a struct in the following way: defmodule MyStruct do # Both x and y will have the FIXED values unti...
New
superchris
Hello! After seeing a couple of posts from Chris and Jose about supporting web components in LiveView, I thought it would be great to ini...
New
sevensidedmarble
Hello all, Apologies if this has been proposed before I guess, but I have a very simple one: With the increasing importance of LV, I th...
New
nunobernardes99
On 1.8+, when we generate an authentication system with mix phx.gen.auth we can make use of magic link login which is amazing and a great...
New
dogweather
Hi Phoenix community! First off, huge thanks for an amazing framework - Phoenix has been a joy to work with. I have a small UX suggestio...
New
eagle-head
Hi everyone, I’ve been researching Content Security Policy Level 3 support in Phoenix and wanted to share my findings and a proposal for...
New

Other popular topics Top

stefanchrobot
What’s the safe way to decode a JSON string into a struct? I want to avoid calling String.to_atom. Jason.decode can give me a map with st...
New
JeremM34
Hello, how can I check the Phoenix version ? Thanks !
New
pmjoe
I have a relationship of love and hate with Elixir. Lots of things are just absolutely right, but there are some things that are kind of ...
New
chrismccord
This release brings a number of exciting features, including integration with the new Phoenix LiveDashboard and Phoenix LiveView. There h...
New
RisingFromAshes
I've read in another post that it may be possible with a router helper - but I couldn't find an appropriate one, and tbh, I'm still just ...
New
romenigld
I am trying to run a deploy with docker and I successfully runned with this command: docker build -t romenigld/blog-prod . but when I t...
New
AstonJ
We’ve put together this wiki for Phoenix LiveView - please feel free to add any info you feel is worth including. What is Phoenix LiveV...
New
WestKeys
Currently suffering from paralysis by [HTTP client] analysis. This is rather unusual in Elixirland as there tends to be consensus on the ...
New
openscript
Hello! Sorry for this astonishing simple question, but I’m really stuck. I try to set up the intellij-elixir plugin, but I don’t know ho...
New
vonH
In asking this question I am more interested about the expressiveness of the language itself and less concerned about the availability of...
New

We're in Beta

About us Mission Statement