Newbie OTP Pheonix Channels Questions - setting up Genservers running Kafka consumers

Phoenix Forum Questions / Help
1

I am looking to build an Elixir/Phoenix solution. The idea is that I have a Kafka topic that is keyed with a user ID that uses the userID as the key into the topic. What I would like to do, is setup Genservcers running Kafka consumers on this topic, and then have that Consumer be able to broadcast data to a Channel which will be identified as “topic:userID”. What I am looking for, is there any gotchas I am gonna face?

I am basing a lot of my work on the Book “Functional Web Development with Elixir, OTP and Phoenix” What I have not been able to find is much information about sending messages from one node to another, where all I have is the userID if that process is on another node of the cluster…

Thanks for the info, and links…

1 Like
2

I think instead of building it all from scratch using basic building blocks, I would look at something like broadway that already has a kafka connector

this may cut amount of work you have to do by half, more if you’re starting your adventure with Elixir and phoenix as building these blocks reliably is requiring some experience in how they behave.

But answering your question, if you are using Phoenix PubSub, and the nodes are clustered, you don’t need to do anything special to broadcast the message in a way that it goes cluster-wide.

2 Likes