Does Phoenix js support ArrayBuffer to be send over to Channels?

[antorocz]

I am trying to send an ArrayBuffer over to the channel via channel.push but it always sends and empty json instead of the ArrayBuffer. I was using socket.io to achieve this as it supports ArrayBuffer and Blob data but since Phoenix recommends using Phoenix.js I switched over but cant seem to send the ArrayBuffer or Blob data to channel. Is this possible or not sure whether I am doing something wrong.

[josevalim]

I believe you can create your own transports/serializzers in Phoenix Channels, that sends the data in a format of your choice (for example, you can replace JSON by MessagePack). You will also need a custom serializer on the server to receive said data. I don’t have a reference in mind, but I hope this gives some direction (or maybe someone will complement it).

[antorocz]

One way that I found was sending as data string and it worked but was costly and became worse as the size increased whereas socketio took the same time with different sizes of data. Was eager to try out Phoenix framework but I guess will have to look at an alternative and come back when Phoenix js supports ArrayBuffer/Blob . Thanks for the help.

[chrismccord]

@antorocz

Was eager to try out Phoenix framework but I guess will have to look at an alternative and come back when Phoenix js supports ArrayBuffer/Blob

As José said, we do support it, but you have to write the serializers and binary protocol yourself. See this post today for an example:

How to disable phoenix channel serialization?

Thanks. follow the idea , here is my hack js client side serializer function toBytesInt32 (num) { var arr = new Uint8Array([ (num & 0xff000000) >> 24, (num & 0x00ff0000) >> 16, (num & 0x0000ff00) >> 8, (num & 0x000000ff) ]); return arr; } function encode(msg, callback){ // check if system message if(Object.keys(msg.payload).length == 0){ let payload = [ msg.join_ref, msg.ref, msg.topic, msg.event, msg.payload] return callback(JSON.stringi…

[antorocz]

@chrismccord That really helped. Just one issue I am facing currently. I am sending message to channel and then waits for acknowledgement like,

channel.push("message:part", {part_num: partNumber, body: data}).receive(
        "ok", (reply) => console.log("part_num ", reply.part_number ," time ", new Date().getTime() - startTime)
       )
       .receive("error", resp => { console.log("ERROR >>>>", resp) })
       .receive("timeout", resp => { console.log("TIMEOUT >>>>", resp) })

For testing I sent like 2000 messages at once and when waiting of the ack I get Timeout after about 1 minute. I also tried setting the timeout to infinity but didnt work

socket "/socket", MyApp.UserSocket,
    websocket: [
      serializer: [{MyAppWeb.Serializers.BlobSerializer, "2.0.0"}],
      timeout: :infinity
    ],
    longpoll: false

[outlog]

clientside or serverside timeout?

@antorocz

channel.push("message:part", {part_num: partNumber, body: data}).receive(

think the default clientside timeout is 10 secs try:
channel.push("message:part", {part_num: partNumber, body: data}, 100000)

[antorocz]

Got it. Now I tried the below,
channel.push("message:part", {part_num: partNumber, body: data}, 180000)
this time I got the timeout log at 3 minutes.

So assuming that I have made 3 push requests,

channel.push("message:part", {part_num: 1, body: data}, 180000) - initiated at 03:01:00

channel.push("message:part", {part_num: 2, body: data}, 180000) - initiated at 03:01:05

channel.push("message:part", {part_num: 3, body: data}, 180000) - initiated at 03:01:10

and each one has a receive callback. So I assume that for the first one I will be getting a timeout at 03:04:00, the second one at 03:04:05 and the third one at 03:04:10 . Is this correct?

[outlog]

@antorocz

Is this correct?

I would very much assume so, yes.

question is why your channel is backed up, does it do processing of the message?

I assume you are uploading something not small - also pretty clear you need to throttle the client so part_num: 2 isn’t sent before part_num: 1 has finished… etc.

[antorocz]

Currently for testing I just write the message to a file. Each part is ~100kb. Also is there a difference between {:reply, {:ok, %{part_number: part_num}}, socket} and push(socket, "completed", %{part_number: part_num}) ?
Tried implementing the second way (push) and listened to the message in client side channel.on("completed", (payload) => {console.log(payload.part_number)}) and also added listeners channel.onError( () => console.log("there was an error!") ) and socket.onClose( (err) => console.log(“the connection dropped”, err) )
Found that sometimes when I run tests with 2000 parts I get the there was an error! and then the connection dropped when the the requests are almost completed and the completed message is not received by the client and the channel join happens again. This happens in like 4 tests out of 10 tests done.

Edit:
Also I have the messages printed in console when each push from server is sent. When the error happens I see that the log suddenly stops but when i check the messages in the network tab, the messages are actually coming from the server. Once the test is completed in server i get the there was an error! in channel.onError and the channel rejoin happens. Btw the test gets completed at ~50 seconds (the processing of 2000 messages).