How to disable phoenix channel serialization?

Questions / Help
1

Hi all,

 I am building a chat app use phoenix framework.

the web client and server send blob to each other, and I dont need the data serialization part.
so the question is more like how can send and receive raw websocket data?

I am quite new to Elixir and Phoenix, just finish the book Programing Phoenix 1.4.
Thanks.

the blob data from web client look like below:

var uint8array = new TextEncoder("utf-8").encode("hallo");
var arrayBuffer = uint8Array.buffer;
var blob        = new Blob([arrayBuffer]);
2

Phoenix doesn’t do generic websocket connections. It’s handling phoenix channel connections via multiple swapable transport layers, where websockets is just one of the possible transports.

If you want just plain websockets you’d probably need to write your own websocket handler for cowboy directly.

1 Like
3

Actually you can use any format of data, but then you need to implement Phoenix.Socket.Serializer on your own. So if you want “raw” sockets then you can use something like:

defmodule MyApp.Serializer.Raw do
  @behaviour Phoenix.Socket.Serializer

  def decode!(iodata, _options) do
    %Phoenix.Socket.Message{payload: List.to_string(iodata)}
  end

  def encode!(%{payload: data}), do: {:socket_push, :binary, data}

  def fastlane!(%{payload: data}), do: {:socket_push, :binary, data}
end
4

Then you’re still implementing channels though, not plain websockets. Things like replies are not a thing for websockets, but are part of phoenix channels.

5

Thanks. This can be a solution for me.
Can I ask why use List.to_string(iodata) in decode!/2?

6

To make it easier to work with, as iodata is useful when writing, but not really when reading.

7

hauleth is right – you can indeed send binary data over channels by implementing your own serializer on the client and server. For example, we use this approach for the LiveView upload feature. To close the gap for @max-vc, and what LostKobrakai was getting at, is you need to handle both control messages and your blob messages. So your custom serializer will need to handle both, ie it can delegate to the default JSON serializer for non binary messages, and pattern match on events/shapes of payloads to handle the blobs. You can also make the protocol entirely binary, but you’ll need to do a lot more work to handle the channel wire format to accommodate refs/acknowledgments/etc. So the mixed use is probably what you want.

2 Likes
8

Thanks. follow the idea , here is my hack

js client side serializer

function toBytesInt32 (num) {
  var arr = new Uint8Array([
       (num & 0xff000000) >> 24,
       (num & 0x00ff0000) >> 16,
       (num & 0x0000ff00) >> 8,
       (num & 0x000000ff)
  ]);
  return arr;
}

function encode(msg, callback){
    // check if system message
    if(Object.keys(msg.payload).length == 0){
      let payload = [ msg.join_ref, msg.ref, msg.topic, msg.event, msg.payload]
      return callback(JSON.stringify(payload))
    }
    else
    {
      var enc = new TextEncoder();
      let pl = [ msg.join_ref, msg.ref, msg.topic, msg.event, {}]
      let json_buf = enc.encode(JSON.stringify(pl));

      let head = enc.encode("{=+}");
      var data_length = toBytesInt32(msg.payload.length);
      let d = new Uint8Array(head.length +data_length.length+msg.payload.length+json_obj_buf.length);
      d.set(head);
      d.set(data_length,head.length);
      d.set(msg.payload,head.length+d_ua.length);
      d.set(json_buf,head.length+data_length.length+msg.payload.length);
      return callback(uint8ArrayP);
    }

function arrayBufCompare(buf1,buf2)
{
  if (buf1.length != buf2.length) return false;
  for (var i = 0 ; i < 4 ; i++)
  {
      if (buf1[i] != buf2[i]) return false;
  }
  return true;
}
function decode(rawPayload, callback){
  var dec = new TextDecoder("utf-8");
  var enc = new TextEncoder();
  var expect_head = enc.encode("{=+}");
  var head_buf = new Uint8Array(rawPayload,0,4);
  if(arrayBufCompare(head_buf,expect_head))
  {
    var data_size_buf = new Uint8Array(rawPayload,4,4);
    let buffer = Buffer.from(data_size_buf);
    var data_size = buffer.readUIntBE(0, buffer.length);
    var data_buf = new Uint8Array(rawPayload,8,data_size);
    var json_buf = new Uint8Array(rawPayload,8+data_size,rawPayload.byteLength - 8 - data_size);
    var json_str = dec.decode(json_buf);
    let [join_ref, ref, topic, event, payload] = JSON.parse(json_str);
    payload["mydata"] = data_buf
    return callback({join_ref, ref, topic, event, payload})
  }
  else
  {
    let [join_ref, ref, topic, event, payload] = JSON.parse(dec.decode(rawPayload))  
    return callback({join_ref, ref, topic, event, payload})
  }
}

let socket = new Socket("/socket", {params: {token: window.userToken}, 
encode: encode, decode:decode, binaryType:'blob'})

custom serializer.ex

def decode!(iodata, _options) do
    <<head::binary-size(4),data_size_b::binary-size(4),_::binary>> = iodata
    if head == "{=+}" do
      data_size = :binary.decode_unsigned(data_size_b)
      <<_::binary-size(4),_::binary-size(4), my_data::binary-size(data_size),json_pkg::binary>> = iodata
      [join_ref, ref, topic, event, _] = Phoenix.json_library().decode!(json_pkg)
      %Phoenix.Socket.Message{
        topic: topic,
        event: event,
        payload: my_data,
        ref: ref,
        join_ref: join_ref
      }
    else
      [join_ref, ref, topic, event, payload | _] = Phoenix.json_library().decode!(iodata)
      %Phoenix.Socket.Message{
        topic: topic,
        event: event,
        payload: payload,
        ref: ref,
        join_ref: join_ref
      }
    end
  end

def encode!(%Phoenix.Socket.Reply{} = reply) do
    data = [
      reply.join_ref,
      reply.ref,
      reply.topic,
      "phx_reply",
      %{status: reply.status, response: reply.payload}
    ]
    {:socket_push, :binary, Phoenix.json_library().encode_to_iodata!(data)}
  end

def encode!(%Phoenix.Socket.Message{} = msg) do
    # I only boradcast message, so here only push system message
      data = [msg.join_ref, msg.ref, msg.topic, msg.event, msg.payload]
      {:socket_push, :binary, Phoenix.json_library().encode_to_iodata!(data)}
end

def fastlane!(%Phoenix.Socket.Broadcast{} = msg) do
    my_payload = Map.get(msg.payload, :payload)
    empty_payload = %{}
    data_empty = [nil, nil, msg.topic, msg.event, empty_payload]
    data_b = Phoenix.json_library().encode_to_iodata!(data_empty)
    data_buf = IO.iodata_to_binary(data_b)
    data_size = byte_size(my_payload)
    data_size_bs = <<data_size::size(32)>>
    head = "{=+}"
    pkg = <<head::bitstring, data_size_bs::bitstring, my_payload::bitstring, data_buf::binary>>
    {:socket_push, :binary, pkg}
  end
1 Like