Using channel and topics - better one topic or 10K topics

Questions / Help
1

Hi,
I get by websocket Json data . I have 10K different packs, but the number of this different kind of packs is constant ( 10k) .
E.x of receiving
[pack1]
[pack2]
[pack3]
…
[pack1]
[pack3]
[pack2]
…
[pack10k]
[pack5]
[pack99]

Via websocket I receive in one second about 1K different packs .
I have 100 RECEIVERS of this packs and each receiver has Agent dedicated to specific packs .

RECEIVER 1
----- AgentForPack1
----- AgentForPack2
…
----- AgentForPack10K

RECEIVER 2
----- AgentForPack1
----- AgentForPack2
…
----- AgentForPack10K
RECEIVER 100
…

Each pack must be calculated by each receiver (by Agent from Receiver dedicated to specific pack, then Agent holds new calculated state) .

Question is : which way is better to code this problem

SOLUTION ONE :
One channel:onetopic which spreads out each PACK to each RECEIVER, then each receiver does some calculation on this pack (via Agent)
SOLUTION TWO :
channel:10Ktopics (each topic is dedicated to specific PACK) , then each receiver is subscribed to 10K channel:10Ktopics

Thanks in advance for answer

1 Like
2

Why are you using channels for this. Could you just register each Agent (or Receiver) to a registry by pack name and send them the packs directly ? (That would be shaped as a 10K topics solution).

Do you have multiple websockets connexion ?

Sorry if what I say makes no sense but when you talk about Agent it seems to be an Elixir Agent, which is server side.

3

Without knowing your specific use case, I’m not sure I’d recommend 10k channels (specific topics). Each channel is a process on your server, so 1 client takes up 10k processes with that approach. I don’t mind throwing a handful of processes to a client, but 10k may be problematic. (It also may be fine—I haven’t benchmarked it)

1 Like