Plan old websockets in phoenix but without magic

Phoenix Forum Questions / Help
1

Hi, I’m looking for a way to create a good old web socket and to send data from and to the browser. The result is going to end up in xterm.js (browser-based terminal), so the whole channels thing is out of the question (I don’t want to re-write xterm.js just to support channels).

I’ve tried phx_raws, but Phoenix still wants to mess with the messages the process gets and breaks stuff.

I’ve looked at overwriting HTTP: on the config, but that breaks the live reloader.

I’m quite lost here, I just want a good old WebSocket, the logic is super simple: if it receives {:binary, bla} from the socket, it calls a library, if it receives {:data, bla} it sends it to the socket. A bit auth sprinkled on it, and that’s it (literally a copy of this https://gitlab.com/Project-FiFo/FiFo/wiggle/blob/test/src/wiggle_console_h.erl).

Any advice, am I missing a flag to turn off ‘magic’?

2 Likes
2

Phoenix’s websocket support is via an abstraction of ‘Channels’ and phoenix does not really have any websocket helpers itself.

Rather what you want, if you want raw websockets, is just to use the cowboy API underneath Phoenix directly. Phoenix tends to not rewrite anything into itself if one of it’s dependencies already handles it, and as cowboy handles raw websockets already then it does nothing special with it, just implement cowboy’s API and put cowboy in your supervision tree, hooking in phoenix in the right place too. There are docs, hmm… somewhere on how to do it, maybe someone will link them? Hard for me to google at moment… ^.^;

2 Likes
3

Thanks mate, yes I looked at the cowboy stuff, https://github.com/phoenixframework/phoenix/issues/234 mentions dispatch_option which seems to no longer exist. The only thing I’ve found is completely hand crafting the dispatch rules which seems to be badly documented and blows up in dev as there are some secret endpoints added :frowning:

4

Yep that’s it, and yep you have to be sure to add in Phoenix’s stuff back again, can just mostly copy/paste it’s setup code though (and add your own). ^.^;

It would be nice if Phoenix’s endpoint setup had a method to add extra things to the cowboy setup… Maybe PR it? :slight_smile:

5

the problem is that approach is not working. Not all required endpoints are not documented and even searching the entire repo doesn’t show where they come from :confused:

6

Well what you ‘usually’ put in your application is:

      # Start the endpoint when the application starts
      supervisor(MyServerWeb.Endpoint, []),

And your Endpoint uses Phoenix’s Endpoint, so checking it’s use function is:

And the endpoint module behaviour defines init and other such interesting things, which delegate to the Phoenix.Endpoint.Supervisor, and here is the interesting part there:

And the interesting part is probably server_children at:

But it becomes hairy, so maybe wait for a Phoenix person like @chrismccord or so? ^.^;

But overall you could get the supervisor data like that then mutate it to add your own part, or maybe Phoenix has a way to add in custom cowboy handlers yet? :slight_smile:

7

The cowboy handler docs allow you to set up a cowboy websocket handler to use as you see fit:

https://github.com/phoenixframework/phoenix/blob/master/lib/phoenix/endpoint/cowboy_handler.ex#L8-L52

You may have tried these docs as you said you have issues but it’s not clear what problems you’re encountering. Note the caveats in the docs where these are cow1 specific and changes here not subject to semver.

8

Have you tried this? This allows you to specify custom Cowboy dispatch list, and then you should be able to handle websocket connections without using the channels protocol.

1 Like
9

Hi sorry you’re right I was really unclear about the issue. I am a bit confused myself by the whole thing :blush:

I did set up a custom dispatcher based on that docs but the moment I did the logs got spammed by errors:

18:42:19.367 [error] Ranch listener 'Elixir.CloudUiWeb.Endpoint.HTTP' terminated with reason: {{#{'__exception__' => true,'__struct__' => 'Elixir.Phoenix.Router.NoRouteError',conn => #{'__struct__' => 'Elixir.Plug.Conn',adapter => {'Elixir.Plug.Adapters.Cowboy.Conn',{http_req,#Port<0.35209>,ranch_tcp,keepalive,<0.854.0>,<<"GET">>,'HTTP/1.1',{{127,0,0,1},65375},<<"localhost">>,undefined,4000,<<"/phoenix/live_reload/socket/websocket">>,undefined,<<"vsn=2.0.0">>,undefined,[],[{<<"host">>,<<"localhost:4000">>},{<<"connection">>,<<"Upgrade">>},{<<"pragma">>,<<"no-cache">>},{<<"cache-co...">>,...},...],...}},...},...},...},...}
[info] GET /phoenix/live_reload/socket/websocket
    [debug] ** (Phoenix.Router.NoRouteError) no route found for GET /phoenix/live_reload/socket/websocket (CloudUiWeb.Router)
    (cloud_ui) lib/cloud_ui_web/router.ex:1: CloudUiWeb.Router.__match_route__/4
    (cloud_ui) lib/phoenix/router.ex:303: CloudUiWeb.Router."call (overridable 2)"/2
    (cloud_ui) lib/plug/error_handler.ex:64: CloudUiWeb.Router.call/2
    (cloud_ui) lib/cloud_ui_web/endpoint.ex:1: CloudUiWeb.Endpoint.plug_builder_call/2
    (cloud_ui) lib/plug/debugger.ex:99: CloudUiWeb.Endpoint."call (overridable 3)"/2
    (cloud_ui) lib/cloud_ui_web/endpoint.ex:1: CloudUiWeb.Endpoint.call/2
    (plug) lib/plug/adapters/cowboy/handler.ex:15: Plug.Adapters.Cowboy.Handler.upgrade/4
    (cowboy) /Users/heinz/Projects/fifo/core/cloud_ui/deps/cowboy/src/cowboy_protocol.erl:442: :cowboy_protocol.execute/4

I interpret this as the live reload server endpoint missing, I hoped the :_ case would provide that but it seems it’s not and I couldn’t figure out where in the phoenix code it was defined to add it like the /socket/websocket endpoint

10

Would you mind sharing in what way Phoenix interferes and breaks your code? I just saw your issue, did you get to configure it? I’m wondering so I can tackle issues down before working on this.

Going with a manual Cowboy configuration otherwise is as well a good option :slight_smile:

11

Sure! This involves a bit of guess work about how phoenix works as it’s really a big black box to me.

The problem, from what I tell, is that Phoenix’s Socket will interpret any message that arrives at the Socket process and tries to handle it.

Simplified what I try to write is a websocket proxy. Data that comes from a process and gets send to the browser. Data that comes from the browser gets send to a process.

The issue start that when the process sends it data to the Socket, the socket treats it the same way it would treat websocket data, meaning it’ll start trying to apply opcodes and things to id.

So when the process sends socket_pid ! {:data, some_binary} then the socket crashes as Phoenix tries to parse that as something coming from the web socket.

1 Like
12

From the example, adding .Raw to the transport is the only change you need to do in order to get it working - in comparsion to a regular socket setup (with channels) one would have. I assume you have this.

We have discussed changing the syntax here, but for now if you want to send data to the client you may do send socket_pid, {:binary, some_binary} or have this tuple returned on the handle callback - as shown in the example. Have a look here for a reference on op codes.

Would you mind sharing your code as well?

13

Sadly that doesn’t solve the problem, I used that and I kept getting the following error:

[error] Ranch protocol #PID<0.514.0> (:cowboy_protocol) of listener CloudUiWeb.Endpoint.HTTP terminated
** (exit) an exception was raised:
** (FunctionClauseError) no function clause matching in :cowboy_websocket.websocket_opcode/1
    (cowboy) /Users/heinz/Projects/fifo/core/cloud_ui/deps/cowboy/src/cowboy_websocket.erl:652: :cowboy_websocket.websocket_opcode(:data)
    (cowboy) /Users/heinz/Projects/fifo/core/cloud_ui/deps/cowboy/src/cowboy_websocket.erl:699: :cowboy_websocket.websocket_send/2
    (cowboy) /Users/heinz/Projects/fifo/core/cloud_ui/deps/cowboy/src/cowboy_websocket.erl:618: :cowboy_websocket.handler_call/7
    (phoenix) lib/phoenix/endpoint/cowboy_websocket.ex:49: Phoenix.Endpoint.CowboyWebSocket.resume/3
    (cowboy) /Users/heinz/Projects/fifo/core/cloud_ui/deps/cowboy/src/cowboy_protocol.erl:442: :cowboy_protocol.execute/4
16:25:51.313 [error] Error in process <0.514.0> with exit value:
{function_clause,[{cowboy_websocket,websocket_opcode,[data],[{file,"/Users/heinz/Projects/fifo/core/cloud_ui/deps/cowboy/src/cowboy_websocket.erl"},{line,652}]},{cowboy_websocket,websocket_send,2,[{file,"/Users/heinz/Projects/fifo/core/cloud_ui/deps/cowboy/src/cowboy_websocket.erl"},{line,699}]},{cowboy_websocket,handler_call,7,[{file,"/Users/heinz/Projects/fifo/core/cloud_ui/deps/cowboy/src/cowboy_websocket.erl"},{line,618}]},{'Elixir.Phoenix.Endpoint.CowboyWebSocket',resume,3,[{file,"lib/phoenix/endpoint/cowboy_websocket.ex"},{line,49}]},{cowboy_protocol,execute,4,[{file,"/Users/heinz/Projects/fifo/core/cloud_ui/deps/cowboy/src/cowboy_protocol.erl"},{line,442}]}]}
16:25:51.313 [error] Ranch listener 'Elixir.CloudUiWeb.Endpoint.HTTP' terminated with reason: no function clause matching cowboy_websocket:websocket_opcode(data) line 652

Since phoenix tries to put the first argument thorugh webxocket_opcode/1

14

I see that error coming from Cowboy itself, because there’s no function clause matching in websocket_opcode(:data). Which I assume is caused by that atom not being a registered op code (text | binary | ping | pong), I guess :binary should work just fine. :slight_smile:

15

it might but that’d require rewriting the library sending {:data, ...} the data and every other applicaiton using it. Not to mention that it’s a horrible hack. I mean it’s not sane that :binary means data send to the socket and :text means data from the socket just because well, because otherwise it explodes.

16

Do you mean that incoming data (that-is data received from the client) is sent over to the handle function with a :text op code only? And that outgoing data (that-is data sent to the client) is to be set as :binary?

That is not the case. You may receive both :text and :binary, as well as :ping and :pong from the client. Thus you may as well send either of those to the client.

def handle(opcode, data, state) do
  case opcode do
    :text ->
      IO.inspect "text!"
      IO.inspect data
    :binary ->
      IO.inspect "binary!"
      IO.inspect data
    :ping ->
      IO.inspect "ping!"
    :pong ->
      IO.inspect "pong!"
    :closed -> # not an actual opcode, this is a 'somewhat custom' phx_raws event
      IO.inspect "rip"
      IO.inspect data # reason
  end

  send self(), {:text, "Hello user!"}
  send self(), {:binary, ...}
  send self(), {:ping, ...}
  send self(), {:pong, ...}
  send self(), {:close, "Goodbye!"}

  :ok
end

For instance, the readme example is an echo server. Upon connection, a Welcome! text message is sent to the client. And whenever a text message is received from the client, the exact same message is sent back.

17

Nono, I understand how opcode work, I mean that Socket interprets every message the process gets as an opcode.

I think I’m really bad at explaining it, let me try with some examples:

We got:

B - A browser (web socket client)

S - The Phoenix.Socket process using transport :websocket, Phoenix.Transports.WebSocket.Raw

E - An endpoint process that somewhere else.

What I try to build is an S that sends all data it gets form E to B and all data it gets form B to E. So basically:

defmodule S do
  use Phoenix.Socket
  transport :websocket, Phoenix.Transports.WebSocket.Raw
  # data coming from B gets send to E
  def handle(:text, message, %{endpoint: endpoint}) do
    FancyLib.send_data(producer, message)
    :ok
  end

  # data coming from E gets send to B
  def handle(:data, message, _state) do
    {:text, message}
  end
end

(semi pseudocodeish)

The problem here is that I can’t find a sane way to send data to S, as all messages that go there get forced through Phoenix.Socket and are interpreted as opcode messages.

18

If I understood you right, this is what we have:

  • A client C sends a message to the server S
  • The server forwards this message to a process P
  • This process does something with the client’s message and sends a reply to the client
    Client > Server > Process > Client

What is FancyLib? Why are you defining handle(:data, ...) - there is no such opcode?

Let me introduce an example.

defmodule Socket do
  use Phoenix.Socket

  transport :websocket, Phoenix.Transports.Websocket.Raw

  def handle(:text, message, state) do
    Something.dothing(message)

    :ok        
  end
end

defmodule Something do
  use GenServer

  def start_link do
    # Start a GenServer on this module and give it a name
    GenServer.start_link __MODULE__, [], name: PotatoServer
  end

  def dothing(message) do
    GenServer.call PotatoServer, {:in, message}
  end

  def handle_call({:in, message}, from, state) do
    # message = message received from the client
    # from = socket PID, the process that called the GenServer
    
    IO.inspect message

    send from, {:text, "I received this: #{message}"}

    {:noreply, state}
  end
end

Say you have a GenServer running named PotatoServer. Following the scheme, we would have a client sending a message to the server, this message forwarded to the GenServer process, and finally this GenServer would send a message back to the client. Client > Server > PotatoServer > Client.

Essentially, by name:'ing a GenServer we are able to refer to it by name instead of using its PID, which is what we are doing when .call'ing.

19

You need to have something like this in you endpoint config:

http: [
    dispatch: [
      {:_, [
        {"/raw_socket", MyAppWeb.RawSocket, []},
        {"/phoenix/live_reload/socket/websocket", Phoenix.Endpoint.CowboyWebSocket,
          {Phoenix.Transports.WebSocket, {MyAppWeb.Endpoint, Phoenix.LiveReloader.Socket, :websocket}}
        },
        {:_, Plug.Adapters.Cowboy.Handler, {MyAppWeb.Endpoint, []}}
      ]}
    ]

Notes:

  • you should only include the second entry (live reload socket route) in :dev environment
  • you might need to include https config as well/instead

Now, you can use the standard cowboy handler, for example:

defmodule CustomDispatchWeb.RawSocket do
  @behaviour :cowboy_websocket_handler

  def init({_transport, :http}, _req, _opts), do:
    {:upgrade, :protocol, :cowboy_websocket}

  def websocket_init(_transport, req, _opts), do:
    {:ok, req, nil}

  def websocket_handle(_msg, req, state), do:
    {:ok, req, state}

  def websocket_info(_msg, req, state), do:
    {:ok, req, state}

  def websocket_terminate(_reason, _req, _state), do:
    :ok
end
8 Likes
20

Thank you :smiley: that does the trick!

2 Likes