Data Serialization in Phoenix Channels

How data is serialized depends on the transport used by the connection. Channels itself are transport agnostic and therefore don’t know how data is handled in flight.

Now u talking about incoming data?
I mean data that i need to send to client?

It’s the same situation in both ways. The transport implementation does handle serialization and deserialization of messages sent between the server and the client.

so as i see, it’s just JSON in both ways by default

Yes, but not arbitrary json, but messages expect a certain format.

Both longpoll and websocket transports of phoenix support two serializer protocols, which you can find here: phoenix/lib/phoenix/socket/serializers at v1.4.8 · phoenixframework/phoenix · GitHub

Hey!
Can i use GitHub - vt-elixir/ja_serializer: JSONAPI.org Serialization in Elixir. · GitHub or GitHub - beam-community/jsonapi: JSON:API Serializer and Query Handler for Elixir · GitHub for data serialization in channels?

Sorry for resurrecting an old thread, but I’m doing this in place of opening an issue on GitHub.

Are the requirements for how to write a client for Phoenix sockets written down anywhere. I also saw another (even older) post on this forum here Specification of the protocol used by Phoenix channels - #10 by LostKobrakai.

My problem is that reading the docs here Phoenix.Socket.Message — Phoenix v1.8.8, it says:

The message format requires the following keys:

:topic - The string topic or topic:subtopic pair namespace, for example "messages", "messages:123"
:event- The string event name, for example "phx_join"
:payload - The message payload
:ref - The unique string ref
:join_ref - The unique string ref when joining

So reading that, it seems like a proper message should be a hash (JSON object) with keys. But this isn’t the case. As noted above, the serializers and the official javascript client both create an array, in a particular order:

let payload = [
      msg.join_ref, msg.ref, msg.topic, msg.event, msg.payload
    ]
    return callback(JSON.stringify(payload))

I came across this issue because I’m using Clojurescript on the client, and was having an impossible time trying to connect re-graph (a graphql client for the re-frame) to Phoenix/Absinthe subscriptions. Apparently the Lacinia server does something completely different, but it took me ages to drill down in the code enough to figure out that I need to send a JSON array, not a JSON object, and then to find exactly the order of the elements. I was so frustrated that I almost considered switching to a Clojure-based backend (but my Java error message PTSD pushed me to try harder to debug this!).

Given that there is no documentation of what is expected to communicate over this channel, I also don’t know the rules behind “ref” and “join_ref”. Can I just make up integers? Should “ref” increment with every new message sent over the channel? It certainly seems from the developer tools console that the Phoenix live code reloading heartbeat increments ref every time. The first message is

["3","3","phoenix:live_reload","phx_join",{}]

and then subsequent messages are:

[null,"4","phoenix","heartbeat",{}]
[null,"5","phoenix","heartbeat",{}]
...

Thanks in advance for pointers of what to read. I’m also willing to write a draft doc, something that could be linked in here: https://github.com/phoenixframework/phoenix/blob/master/guides/realtime/channels.md#client-libraries

There are a few things to unpack: %Phoenix.Socket.Message{} is the datatype used for messages within elixir. What’s send over the wire it actually fully up to the serializer used, while the question of “what is the wire” is solved by the channel transport.

Phoenix actually comes with two serializers: V1 uses json objects, while V2 uses arrays to save bytes (no keys); as well as two transports: long poll and websockets.

There’s already documentation about it here:

Edit: I’m not whining…but re-reading this it reads like I am! I’m just trying to point out some difficulties I’m having navigating this problem, in the hopes that changes can be made that will smooth the path for someone else. I’m fine with working on this until I fix it, as it helps to learn the ins and outs of Phoenix and Elixir.

I’ve read the Phoenix.Socket doc you linked to. Nowhere in that doc does it define the array format expected by the JSON V2 serializer. It does link to docs on Phoenix.Socket.Serializer — Phoenix v1.4.13, which in turn says that JSON V2 is the default serializer, but here is no documentation of the two serializers, nor is there any instruction on how to insert my own serializer. There is only the message: “Custom serializers may be configured in the socket.”

I just grepped my project for V2 and Serializer etc etc, and I can’t see any existing config setting for that, nor do I see it in any of my code. It is like a secret handshake type of thing, which means it should be documented somewhere so that it isn’t so secret anymore. I can’t figure out how to change from V2 to V1 JSON serializer, let alone slot in my own.

I think what I need to do in this case is (A) write a custom Phoenix serializer to properly marshall and unmarshall the messages from re-graph, or (B) write a serializer inside of re-graph that talks JSON V1 or JSON V2. I have no idea how to do (A), so that leaves (B). To do (B), I’m looking for documentation of what the two serializers expect, which at this point is just read the source of the serializers. But the source code doesn’t describe the meaning behind the ref and join_ref fields.

In general, the docs for Phoenix are pretty great in all respects. I just think this is a gap that should be filled in some way.

It’s not clear what your goals are on the clojurescript side. Are you using a phoenix channels js client (ie phoenix.js) or are you trying to implement a channels client entirely on the clojurescript side? Merely shoving data on the wire on the websocdket connection won’t be enough, so it’s not clear what gaps you are trying to fill from the regraph => channel server. Since you are using Absinthe subscriptions, they have a javascript library which wraps phoenix.js: absinthe-socket/packages/socket at master · absinthe-graphql/absinthe-socket · GitHub

Is that not callable from clojurescript? I’m not at all familiar with regraph, but if your goals are to take client data from regraph and send it up to absinthe, I think the existing js has everything you need other than glueing the regraph object to the absinthe js client API?