WebSockex - An Elixir WebSocket client

[Azolo]

Hey everyone,

I just released WebSockex which is a Elixir WebSocket client.

WebSockex strives to work as a OTP special process, be RFC6455 compliant, and be simple to use by providing smart defaults for common actions.

Take a look and let me know about any questions, suggestions, or any other comments you may have,

Azolo/websockex

[Azolo]

Currently SSL peers aren’t verified. I tried to figure out how to make it work, but it seems really DIY in the underlying Erlang :ssl module.

If anyone has any advice on how to handle this it would be much appreciated.

[jackalcooper]

Good library. I like the idea to map WS to a process. The WS library I’m using now doesn’t provide process abstraction so I have to implement it every time.

[ericTsiliacos]

Does your library support reconnect? When the server goes down, we’d like the websocket client to try to reconnect indefinitely.

[Azolo]

It does, take a look at the docs for handle_disconnect.

[hildersantos]

Thanks for sharing this with the community. I was looking for something exactly like this - I’m developing some API clients right now, and one of them needs to connect with a WebSocket. I guess this will help me a lot!

I’ll test it on the coming days.

Thanks again!

[Jonathan]

Awesome work, Im having some trouble though, how do i send json, im currently trying Client.send_frame(pid, {:text, "{'Page':{'url':'asdf'}}"}) but i get the response `Received Message: {“error”:{“code”:-32700,“message”:“Message must be a valid JSON”}}`` from the connected server… any help would be appreciated

[zambal]

@Jonathan

Client.send_frame(pid, {:text, “{‘Page’:{‘url’:‘asdf’}}”})

JSON should be double quoted, but I would just use a JSON encoder instead of manually writing it out:

json = %{Page: %{url: "asdf"}} |> Poison.encode!()
Client.send_frame(pid, {:text, json})

[Jonathan]

thanks! the double quote did the trick, but yeah, probably better to use an encoder…

[Azolo]

WebSockex 0.1.3 Released!

The ChangeLog for this release includes:

• Client.start_link will no longer cause the calling process to exit on
  connection failure and will return a proper error tuple instead.
• Change WebSockex.Conn.RequestError to WebSockex.RequestError.
• Add handle_connect_failure to be invoked after initiating a connection
  fails. Fixes #5

Checkout the v0.1.2..v0.1.3 diff for more info.

[Azolo]

WebSockex 0.2.0 Released!

This release has some major API changes where callbacks are changed or removed without deprecation.

The affected callbacks are:

• init/2 - Removed in favor of handle_connect/2.
• handle_disconnect/2 - The first argument is now a map.
• handle_connect_failure/2 - Removed with functionality rolled into handle_disconnect/2.

Please don’t be afraid to provide feedback. WebSockex is already meeting my needs, I’d like more information on what needs to change to meet the needs of others.

The full changelog for this release includes:

Major Changes

• Moved all the WebSockex.Client module functionality into the base WebSockex module.
• Roll handle_connect_failure functionality into handle_disconnect.
• Roll init functionality into handle_connect

Detailed Changes

• Roll init functionality into handle_connect
  • handle_connect will be invoked upon establishing any connection, i.e., the intial connection and when reconnecting.
  • The init callback is removed entirely.
• Moved all the WebSockex.Client module functionality into the base WebSockex module.
• Changed the Application module to WebSockex.Application.
• Add WebSockex.start for non-linked processes.
• Add async option to start and start_link.
• Roll handle_connect_failure functionality into handle_disconnect.
  • The first parameter of handle_disconnect is now a map with the keys: :reason, :conn, and :attempt_number.
  • handle_disconnect now has another return option for when wanted to reconnect with different URI options or headers: {:reconnect, new_conn, new_state}
  • Added the :handle_initial_conn_failure option to the options for start and start_link that will allow handle_disconnect to be called if we can establish a connection during those functions.
  • Removed handle_connect_failure entirely.

Checkout the v0.1.3..v0.2.0 diff for more info.

[sasajuric]

This looks pretty interesting, great work!

@Azolo

WebSockex strives to work as a OTP special process

Out of curiosity, why did you roll your own special process, instead of using e.g. GenServer?

[Azolo]

@sasajuric

Out of curiosity, why did you roll your own special process, instead of using e.g. GenServer?

It’s because I’m too curious for my own good.

When I realized, “Oh right, this should play nice with OTP.” I decided to learn what made a process “special” instead of just putting it in a GenServer.

There was a point where I was frustrated and started to roll it into a GenServer, but at that point I also realized the design choice that was frustrating me and changed it.

[idi527]

If it’s not much of a bother, could you briefly explain how it is different from websocket_client [0]?

[0] https://github.com/sanmiguel/websocket_client

[Azolo]

Sure, but first let me say there are two different websocket_client repos that I used.

The first one was the one listed on hex.pm(which is the one you linked). It implements gen_fsm and it more actively maintained. However, there are also a couple bugs. Most of them were already reported issues, but one in particular was that terminate was being called consistently in one of my cases. However it was fickle and I had problems reproducing it, but it was causing my tests to fail 1/15 times. So when I tried to jump and trace it, I was overwhelmed by the sheer number of things the repo was trying to do.

So I looked at the original fork. It didn’t have any of the bugs I experienced but also wasn’t an OTP special process and didn’t wait for the server to close the socket (as per the spec). I think I also had some problems with close frames that contained a payload. But this repo is no longer maintained in favor of the one above.

So, I decided to to write my own. I wanted it to be some where in between. Being compatible with OTP, but simple enough that someone could jump into and be able to submit simple bug-free code without being overwhelmed.

Also, WebSockex is has great test coverage.

[michalmuskala]

The remark that it implements special process by hand made me interested, so I had a look at the code. Here are some things I spotted, I hope you don’t mind.

• you don’t handle the regular naming conventions - it would be nice to be able to pass the name option like with all the Elixir’s stdlib behaviours. When you do this, there are some race conditions to consider, so I would advise looking at using the :gen module that powers all the Erlang behaviours (it’s not documented publicly, but I saw many projects using it, I’m not sure how bad of an idea that is - but it definitely handles a lot of common scenarios).
• whenever you call user module callbacks, they should be wrapped in a try and errors should be handled appropriately (e.g. you want to call the terminate callback in that case before dying).
• it’s nice to use :sys.handle_debug for any messages received and sent out (or any “major” events that happen in a process) - this is really useful when things go wrong.
• in the receive loop, you probably want to look for the {:EXIT, parent, reason} message in case the user starts trapping exits in the process. The special process has to exit with the same reason as the parent in case that happens.
• it looks like you don’t implement the system_code_change/4 callback
• there’s an additional format_status/2 callback you can implement to control what information is returned on :sys.get_status and :sys.get_state used, for example, by observer.

[sasajuric]

@Azolo

It’s because I’m too curious for my own good.

When I realized, “Oh right, this should play nice with OTP.” I decided to learn what made a process “special” instead of just putting it in a GenServer.

I absolutely appreciate and encourage trying different things, and even overcomplicating the code for the purpose of learning and gaining a deeper understanding. That being said, if learning is the only motivation for using your own special process here, and assuming that this library is meant to be used by others and contributed to, I think it might be wise to switch to GenServer. Doing this would make it easier to solve (and in some cases automatically solve) issues mentioned by @michalmuskala.

It might also simplify some future additions. For example, I think that calls should be supported as well, and this would be pretty simple if the process was a GenServer. Making the interface of your behaviour more similar to GenServer by supporting timeouts and hibernate options in return tuples, as well as stop option would also be nice. Again, this should be fairly trivial if your behaviour was powered by GenServer.

@Azolo

The first one was the one listed on hex.pm(which is the one you linked). It implements gen_fsm and it more actively maintained. However, there are also a couple bugs. Most of them were already reported issues, but one in particular was that terminate was being called consistently in one of my cases. However it was fickle and I had problems reproducing it, but it was causing my tests to fail 1/15 times. So when I tried to jump and trace it, I was overwhelmed by the sheer number of things the repo was trying to do.

I have similar sentiments about the current state of WS clients. About a year ago, I looked into a couple of them, and wasn’t really completely pleased with any. In the end, I settled for this branch. I like that your project starts with a clean slate, and has good test coverage. That seems very promising!

[Azolo]

Awesome feedback, thanks a ton. A couple of other things I didn’t mention or am starting to remember now.

@michalmuskala

you don’t handle the regular naming conventions - it would be nice to be able to pass the name option like with all the Elixir’s stdlib behaviours. When you do this, there are some race conditions to consider, so I would advise looking at using the :gen module that powers all the Erlang behaviours (it’s not documented publicly, but I saw many projects using it, I’m not sure how bad of an idea that is - but it definitely handles a lot of common scenarios).

So the :gen module implements the call behavior. Which I had a really long thought process where I didn’t really want to have any synchronous calls because of the behavior of WebSockets and the way reconnects are implemented. That may change in the future but for now I don’t want any thing that can Timeout.

But I don’t mind adding a :name option. That actually doesn’t look too hard.

In the :gen module it looks like it checks during start and then again during the init function when it tries to register the process.

@michalmuskala

whenever you call user module callbacks, they should be wrapped in a try and errors should be handled appropriately (e.g. you want to call the terminate callback in that case before dying).

Ahh, good point. I did that for some of them, but I don’t think that it’s tested and it isn’t consistent.

@michalmuskala

it’s nice to use :sys.handle_debug for any messages received and sent out (or any “major” events that happen in a process) - this is really useful when things go wrong.

Yeah, that is me being lazy. I also need to add an error message on exit.

@michalmuskala

in the receive loop, you probably want to look for the {:EXIT, parent, reason} message in case the user starts trapping exits in the process. The special process has to exit with the same reason as the parent in case that happens.

I totally didn’t know that. Thanks.

@michalmuskala

it looks like you don’t implement the system_code_change/4 callback

Oops. Thanks, incidentally I have no idea how to test this.

@michalmuskala

there’s an additional format_status/2 callback you can implement to control what information is returned on :sys.get_status and :sys.get_state used, for example, by observer.

This is me being lazy again.

Thanks again for the advice!

[Azolo]

@sasajuric

Making the interface of your behaviour more similar to GenServer by supporting timeouts and hibernate options in return tuples, as well as stop option would also be nice.

This made me remember why I didn’t use GenServer. I didn’t want those behaviors to be available because it could mess the handling of the control frames and I didn’t want to move the WebSocket into another child process.

I had the idea of writing another module based off GenServer that would wrap a WebSockex connection. This would make doing complex things like synchronous sends possible without messing too much with the flow that the WebSocket spec asks for.

[michalmuskala]

@Azolo

So the :gen module implements the call behavior. Which I had a really long thought process where I didn’t really want to have any synchronous calls because of the behavior of WebSockets and the way reconnects are implemented. That may change in the future but for now I don’t want any thing that can Timeout.

The fact that call is synchronous for the caller, does not mean it’s synchronous for the server. Using an explicit GenServer.reply/1 you can reply to the message before returning from the handle_call function, after or even from a completely different process.