azyzz228

azyzz228

Network optimization: 4x WS message size reduction when `push_event`

The slow network is known to be an Achilles heel of LiveView’s architecture.

Recently, I was working on creating a fast rendering map with 12,000+ points that are coming from DB. My server sent data in chunks of 4000 and it worked great on my localhost. But when deployed, network made it super slow.

I was able to reduce the WS message size with my data by 4 times (from 400kb to just 100kb) by using compression technique:

On the Elixir side,

  defp compress(data) do
    data
    |> Jason.encode!()
    |> :zlib.compress()
    |> Base.encode64()
  end

and on the JS side:

const decode_base64_decompress_to_json = (base64string) => {
    let base64_decoded = Base64.atob(base64string)
    let charCodes = [];
    for (let i = 0; i < base64_decoded.length; i++) {
        charCodes.push(base64_decoded.charCodeAt(i));
    }
    let inflatedData = pako.inflate(charCodes, { to: 'string' });

    const data = JSON.parse(inflatedData);
    return data;
}

I think this technique can drastically improve communication between the LiveView process and client-side JS when both are passing medium to large amounts of data. I found execution time of both functions + increased bundle size from JS dependencies (pako + Base64) to be negligible compared with the performance boost they generate.

I am happy to work on PR, though will probably need some support.

Most Liked

outlog

outlog

did you try websocket: [compress: true] ? - Phoenix.Endpoint — Phoenix v1.8.8

not sure what the default is.

tj0

tj0

I know this may sound silly, but it could be the way it’s being measured. I know firefox / chrome devtools measures the transferred and total bytes for http, but only do total bytes for websocket on my versions.

My websocket payload for a single request is approximately 90kB as measured by devtools. Measuring network traffic with btop on localhost, the total transfer was 467 kB after roughly 50 requests, so compression is definitely enabled for me.

Endpoint.ex

  socket "/live", Phoenix.LiveView.Socket,
    websocket: [ connect_info: [session: @session_options], log: false, compress: true]


mwhitworth

mwhitworth

Interesting idea! Maybe this and other properties of the LV data flow between client and server could be represented as a pipeline of plugs

When you talk about a “fast rendering map”, what is this exactly?

One point of view might be that there is a rough upper limit on the amount of elements visible on the screen at any one time (and likely to change), and so techniques that take into account the current client viewport somehow might be one way to cut down on the traffic

(I haven’t seen this done though!)

Where Next?

Popular in Proposals: Ideas Top

rmoorman
Current situation Currently, the structure of the HTML returned by phoenix is determined by the layouts (components/layouts/[root,app].ht...
New
shahryarjb
When proposing or suggesting something please consider: As my experience implementing getBoundingClientRect as Phoenix.LiveView.JS funct...
New
dogweather
Hi Phoenix community! First off, huge thanks for an amazing framework - Phoenix has been a joy to work with. I have a small UX suggestio...
New
andypearson
Hey all, I have been working on some performance improvements for the Phoenix application I work on. As part of this, through trial and...
New
spicychickensauce
I’ve previously explored what is possible today with hacks to implement view transitions in our apps: I have since created a fork to im...
New
GenericJam
I’ve been messing around with image generation models recently and thought this could be wrapped in a library or people could just use th...
New
woylie
We are seeing a lot of warning logs like this: navigate event to "https://someurl" failed because you are redirecting across live_sessio...
New

Other popular topics Top

rms.mrcs
Hi, I need to transform a list of numbers into a map where the keys are the indexes and the values are the original values of the list. ...
New
JeremM34
Hello, how can I check the Phoenix version ? Thanks !
New
minhajuddin
I have seen a lot of code which picks the first element from a list using Enum.at(0) instead of List.first. Is there a reason why people ...
New
stefanchrobot
What’s the safe way to decode a JSON string into a struct? I want to avoid calling String.to_atom. Jason.decode can give me a map with st...
New
AstonJ
Seen any cool LiveView demos, sample apps or examples? Please post them here! :003:
New
siddhant3030
Hi, I have to write a raw query for one of my project. But till now I have used ecto queries and don’t have much experience writing raw ...
New

We're in Beta

About us Mission Statement