Phoenix Websocket MQTT

Phoenix Forum Questions / Help
1

Hi,

In our company we are working on a contextual awareness mobile SDK, which we want to be able
integrate in other applications.
We also want to be able get “push notifications” inside our SDK (we can’t use the “native” mobile push notification solution) and use as less as possible battery resources.
Our Idea was to use MQTT protocol for that…
My Idea is to convince my bosses to use Elixir/Erlang based solution not only for the push but also as a solution for
RoR scale issues.
I saw the presentation of 2M connections with Phoenix and got really excited about it + the Elixir features and
the promised land of great performance.

I’m wondering If/how can we implement(“use”) MQTT protocol over phoenix sockets/channels or maybe it’s
just better for us to use solution such as EMQTT

Thanks you of answer.

4 Likes
2

You could have a look at VerneMQ, which is an open source MQTT broker (also) build in Erlang, which supports web-sockets directly.

It is also possible to write plugins for VerneMQ in Elixir. Notice: This can impact the performance and stability of the broker, because the code is loaded into it. There are some examples out there: https://github.com/larshesel/vmq_elixir_plugin

I’ve wrapped one of their support libraries in Elixir: https://github.com/gausby/gen_mqtt which implements a process behaviour that can connect to-, listen-, and publish to a broker (not just a VerneMQ). It is a matter of implementing a MQTT process that listen to the topics, stick that process in the supervison tree, and use Phoenix broadcast to send messages to the correct recipients—This solution is not perfect as it doesn’t take clusters into consideration; I don’t think I would have more than one listener broadcasting to the Phoenix sockets as the same message would get posted more than once—but gen_mqtt should allow the implementor to deal with these problems (please tell me your thoughts if you go down this route).

3 Likes
3

First of all thank you for your replay!

If I understood you right, what you are basically saying is to use “external” broker - some more dedicated solution, such as VerneMQ or EMQTT and then some “client” to enable communication with the broker - which is totally good solution. But I must say that I’m a bit surprised that non of these
brokers still have their home made elixir SDK’s.
I was also wondering if can turn each Phoenix channel to a kind of broker with single client. (may be it’s total SCI-FI)
Anyway, thank you very much. If we will go with the VerneMQ/EMQTT I probably will try gen_mqtt and will gladly share my experience.

1 Like
4

I think implementing a broker in Phoenix seems like a huge task. I would just go for VerneMQ because it already has the benefits as a broker in Phoenix would have; it can run as a cluster, it has very good throughput.

What do you mean by SDK? In VerneMQ’s case you can build plugins and have it run on the broker. You can do all kinds of things with the subscribe flow, publishing flow, and authentication http://vernemq.com/docs/plugindevelopment/

I need to do all this kinds of things myself, so please get in contact via private message or other means of communication. If you are on IRC you can join #vernemq on Freenode. I am sure Erlio (the company behind VerneMQ) would love to hear your thoughts, and your use-case. I usually hang there trying to push them to do more stuff with Elixir :slight_smile:

5 Likes
5

Gausby, thanks again!

A bit out of scope, just wondering if you have also experience with EMQTT?
VerneMQ looks a bit young for me, did you used it in production?

Regarding the SDK, I meant just a convenient way to post / subscribe to broker.
Basically what you did by yourself - gen_mqtt. BTW the plugins feature of VerneMQ looks very interesting!

Would be great to do some stuff together! Just need to get the OK from the boss :slight_smile:
But I believe it would be also cool as of job task.

1 Like
6

I have no experience with EMQTT, sorry. I picked VerneMQ because it was fairly easy to get started with, and the team is located in the same timezone as me, so they are available during the day. Also, it was fairly easy to connect to the broker from my Elixir application with :gen_emqtt (found in vmq_commons—this is what I later wrapped into gen_mqtt).

It is a young project, but I think it is fairly stable and they have some impressive load tests. Their webpage has pretty good documentation about the broker itself, and I hope that I’ve made some good documentation for the gen_mqtt Elixir wrapper.

Regardless of your choice in message broker you can still use gen_mqtt to subscribe and publish to topics; if you can’t it’s a bug and need to get fixed. I probably can’t share my code (which is in development and about to enter testing before going to production), but I can talk about my thoughts and how I go about architecture.

1 Like
7

Sorry to highjack this thread, but I’m using (trying to use really) GenMQTT to talk to VerneMQ in my Phoenix application. I’m brand new to elixir/erlang/functional programming so this might be like a “doh” question.

I have an IoT Device which connects to VerneMQ. On the other side is the IoT handler who subscribes to the IoT queue. So now the IoT Device publishes a message which is correctly received by the IoT Handler. Depending on the Message the IoT Handler has to send something back to the IoT Device. And this is where everything breaks down. In the on_published callback in IoT Handler I have something similar to

{:ok, pid} = GenMQTT.start_link(IoTHandler, self)
mqtt_channel = "iot/#{iot_id}/#{iot_name}/command"
#Process.sleep(100)
GenMQTT.publish(pid, mqtt_channel, "command to do", 0)

This code only works, meaning the IoT Device only receives the published command, if the Process.sleep(100) is not uncommented. Now I know why this is happening. The start_link function takes some time to negotiate with VerneMQ and connect to it. What I don’t know is how I can utilize the callback function on_publish because I have no idea how to pass variables to the callback. Can this be achieved via the state variable? If so, how?

Thanks

2 Likes
8

Do you have a link to the docs about this on_publish method so we can look at it? :slight_smile:

9

Sure. https://hexdocs.pm/gen_mqtt/GenMQTT.html#c:on_publish/3.

The Code for the on_publish callback is here: https://github.com/gausby/gen_mqtt/blob/master/lib/gen_mqtt.ex#L194

1 Like
10

Yes, this is precisely the use of state, state holds whatever you want. :slight_smile:

11

I can’t seem to figure it out. State is a process. But I can’t set values on it.

12

Looks like a normal state variable that you can have been whatever want? You could make it a record for example, to store multiple different things easily. :slight_smile:

1 Like
13

I might suspect that you have looked at the unit tests for an example? I pass in a pid to a process so I can send it messages when certain things I test for happens. The state can be any term, so tuples, maps, what ever you need :slight_smile:

14

I have. But I’m still completely new to Erlang/Elixir and this makes it kinda hard. Still I’ll try again :). Thanks for your help (and OvermindDL1).

1 Like
15

Hi,

I’m starting to try this out. Is there a good tutorial for this? I tried both the phoenix_pubsub_vernemq and gen_mqtt and not sure how to use and connect it from phoenix. :frowning:

Thanks!