joshp

joshp

Correct way to structure PubSub broadcast data from a library

Hey all!

I’m building a library which will be listening to UDP events broadcast on the network, doing some parsing on them, and emitting the parsed events as Phoenix.PubSub broadcasts. The underlying UDP data has a number of different event types. My goal is to make it easy to handle all events emitted, or a specific type.
My question is about what the correct way to structure the pubsub broadcast to accommodate this is.

The typical PubSub structure is

PubSub.broadcast(:my_pubsub, "user:123", {:user_update, %{id: 123, name: "Shane"}})

My first pass was to do something like

PubSub.broadcast(:my_pubsub, "libraryname:udp", {:event_type_name, %{id: 123, data: "goes here"}})

But the problem is that when I subscribe to the “libraryname:udp” channel, I then need to pattern match for every single {:event_type_name, event} that might be broadcast, and have no way to easily pattern match for any event. (No way to differentiate an {:event_type_name, _} broadcast by my library from an {:event_type_name, _} broadcast by something else, unless I inject the topic name into the payload and match against that too, which seems inelegant.)

So my next attempt was to mimic the %Phoenix.Socket.Broadcast{} struct, and do this:

obj = %{topic: "libraryname:udp", event: :event_type_name, payload: %{id: 123, data: "goes here"}
PubSub.broadcast(:my_pubsub, "libraryname:udp", obj)

which allows users of the library to easily pattern match for all messages with:

def handle_info(%{topic: "libraryname:udp", event: event_type, payload: payload}, socket) do
    Logger.debug("Got event type #{event_type}, with data: #{payload}")
    {:noreply, socket}
end

or further match for specific event types by matching against the event type as well as the topic.

Now for my case, as a user of the library I’m writing, this is preferable! If I weren’t publishing the library I’d just go with it and not care. However I’m curious if this would be considered poor form for a library.

If the only thing that matters is documenting your output, then great, but I have the feeling I might be violating a norm by emitting events over a Phoenix.PubSub in an unexpected format, and I’d prefer not to have to re-write the library later with a breaking change just to do it the Right Way™. :slight_smile:

Marked As Solved

lud

lud

what if multiple processes need to consume the data

What if one consumer is a java app athat needs the even in kafka?

And what if you do not want a bottleneck gen server for the latest data but an ETS table ?

You can never cover all the cases I guess, so the best is to not close doors.

If you really want your library to force usage of Phoenix.PubSub then maybe accepting an option for the topic is ok. Or a topic prefix maybe. And then a tuple {YourLibrary, :actual_name} as the event name could work, so if the user provides a topic that other publishers publish to, there is still a way to match your library’s messages.

Edit; but I would just do both. Accept a callback where the default implementation (if no option given) is a broadcast to Phoenix.PubSub. And pass all the options given to your library when calling the callback.

Also Liked

lud

lud

For a library I think that the best would not to publish to phoenix pubsub at all.

I guess your UDP listener is running is a process, for instance a GenServer using gen_udp. In that case, your start_link/child_spec function should accept an option called event_handler or something like that, and the user can give their own implementation.

There are different ways to do that. The simplest is when a user will give a fun, and you just call the fun with an event like %{event: :some_event, payload: %{id: 123, data: "goes here"}}. Then the user can filter the events if needed, and dispatch to their own pubsub or do whatever they want with it.

You can structure your event data as you want because there is no more norm to violate in this case.

Another possible way is a callback module. You define a behaviour and the user implements that behaviour with their own module and give the module as the option value. So you can define different callbacks like handle_event, handle_closed. This allows the behaviour to define an ìnit function so the user can return an initial state that you would pass to further callbacks and update when the callbacks also return a state (like in a GenServer handle_call when you return a new state). This may not be needed at all. Also a fun/2 can do the job, when one argument is the callback type.

Finally, depending on your architecture, the user-defined handler can be a process. Your option can accept a pid but also an atom or a via-tuple so if the user process crashes and is restarted with a new pid but the same name, you can reach it. With a process, you just send your events as messages to that user process.

I’d start with a fun though. It’s simple and goes a long way.

joshp

joshp

Yeah, after thinking about it a bit you make a great point. Locking in a specific way of doing things make it a much less viable library.

Also, doing both and using the tuple event name seems like a pretty perfect solution for all use-cases, I’ll give it a go.

Thanks for the help, really appreciate the ideas!

Where Next?

Popular in Questions Top

siddhant3030
Hi, I have to write a raw query for one of my project. But till now I have used ecto queries and don’t have much experience writing raw ...
New
minhajuddin
I have seen a lot of code which picks the first element from a list using Enum.at(0) instead of List.first. Is there a reason why people ...
New
LegitStack
I’m trying to make a websocket server in Phoenix or raw Elixir. I heard about gun, I think I could use cowboy, but since I’m not that sma...
New
stefanluptak
Hello everybody, usually, I use a 29" ultra-wide monitor for VSCode which can easily accomodate explorer (files panel) + file with code ...
New
RisingFromAshes
I’ve read in another post that it may be possible with a router helper - but I couldn’t find an appropriate one, and tbh, I’m still just ...
New
sergio_101
I am VERY much an elixir newbie. I have taken one elixir course and one phoenix course on Udemy. During that course, I saw the instructor...
New
jason.o
In the code below, if the create action is not set to accept “extra_key” as an input, it errors out with a message shown above. Is there ...
New
WestKeys
Currently suffering from paralysis by [HTTP client] analysis. This is rather unusual in Elixirland as there tends to be consensus on the ...
New
openscript
Hello! Sorry for this astonishing simple question, but I’m really stuck. I try to set up the intellij-elixir plugin, but I don’t know ho...
New
lanycrost
Hi everyone! I need implement if…else if…else condition from my elixir code, and anymore of this control flow structures not work proper...
New

Other popular topics Top

Darmani72
If I have a post route which an argument: post /my_post_route/:my_param1, MyController.my_post_handler How would get the post params ...
New
senggen
Erlang/OTP 25 [erts-13.2.2] [source] [64-bit] [smp:8:8] [ds:8:8:10] [async-threads:1] 15:22:35.803 [error] gen_event {lager_file_backend...
New
lastday4you
I wanted to check elixir version in phoenix because i found that my elixir is 1.5 but when i use Enum.chunk_by it said the function is un...
New
ovidiubadita
Hey all, I discovered Elixir and I love it. I always wanted to learn a functional programming and I intended to go for Haskell, but afte...
New
Qqwy
Original source of discussion: This topic on the Pragmatic Programmers’ Functional Web Development with Elixir, OTP, and Phoenix forum. ...
New
bsollish-terakeet
Credo is smart enough to check for (something like) this: assert length(the_list) == 0 with this response: Checking if an enum is empt...
New
komlanvi
Hi everyone, I was playing with phoenix liveView but I run into an issue. I have a form and want to validate each input text when the te...
New
Brian
What is the proper way to load a module from a file in to IEX? In the python world, doing something like this pretty standard: from ....
New
joaquinalcerro
Hi there, I am working with Ecto-Postgresql and I need to call all of the records from a specific table but the table has 40,000 records...
New
hariharasudhan94
Lets say I have map like this fetching from my database %{"_id" => #BSON.ObjectId<58eb1a7a9ad169198c3dXXXX>, "email" => ...
New

We're in Beta

About us Mission Statement