llama

llama

PubSub, too many subscriptions?

I’m a little hazy on what the best practice is for designing a scalable PubSub system. Should there be a few subscriptions, or even just one subscription, from which relevant information is filtered? Or should there be possibly hundreds or more subscriptions which contain very specific information and thus needs less filtering?

Many specific subscriptions seems like the way to go, but at the same time, having users subscribe to hundreds of topics when they log in also seems like it’d slow things down.

Marked As Solved

ruslandoga

ruslandoga

Note that you can add metadata to the subscriptions that can later be used for filtering during dispatch.

Phoenix.PubSub.subscribe(YourApp.PubSub, "topic:user-id", metadata: [some: :extra_info])

which (if you are familiar with Registry) is similar to

Registry.subscribe(YourApp.PubSub, "topic:user-id", [some: :extra_info])

In general, the number of subscriptions doesn’t affect much, as the total throughput should stay about the same:

bench.exs

Mix.install([:benchee, :phoenix_pubsub])

{:ok, _pid} = Phoenix.PubSub.Supervisor.start_link(name: App.PubSub)

defmodule Sub do
  use GenServer

  def start_link(opts \\ []) do
    GenServer.start_link(__MODULE__, opts)
  end

  @impl true
  def init(opts) do
    topic = Keyword.fetch!(opts, :topic)
    Phoenix.PubSub.subscribe(App.PubSub, topic)
    {:ok, nil}
  end

  @impl true
  def handle_info(_message, state) do
    {:noreply, state}
  end
end

Enum.each(1..200_000, fn _ -> Sub.start_link(topic: "topic") end)
Enum.each(1..10, fn _ -> Sub.start_link(topic: "topic2") end)

map = %{"some" => "key", "then" => "some oter key"}

Benchee.run(
  %{
    "message" => fn topic -> Phoenix.PubSub.broadcast(App.PubSub, topic, "message") end,
    "map" => fn topic -> Phoenix.PubSub.broadcast(App.PubSub, topic, map) end
  },
  inputs: %{"lots subs" => "topic", "few subs" => "topic2"}
)

> elixir bench.exs

Operating System: macOS
CPU Information: Apple M1
Number of Available Cores: 8
Available memory: 8 GB
Elixir 1.13.1
Erlang 24.2

Benchmark suite executing with the following configuration:
warmup: 2 s
time: 5 s
memory time: 0 ns
parallel: 1
inputs: few subs, lots subs
Estimated total run time: 28 s

Benchmarking map with input few subs...
Benchmarking map with input lots subs...
Benchmarking message with input few subs...
Benchmarking message with input lots subs...

##### With input few subs #####
Name              ips        average  deviation         median         99th %
map          147.35 K        6.79 μs  ±1531.35%        4.99 μs       24.99 μs
message       95.00 K       10.53 μs  ±9207.44%        2.99 μs        8.99 μs

Comparison:
map          147.35 K
message       95.00 K - 1.55x slower +3.74 μs

##### With input lots subs #####
Name              ips        average  deviation         median         99th %
map              3.12      320.16 ms     ±9.22%      313.18 ms      387.97 ms
message          2.78      360.19 ms    ±16.91%      352.94 ms      479.31 ms

Comparison:
map              3.12
message          2.78 - 1.13x slower +40.02 ms

Where Next?

Popular in Questions Top

vertexbuffer
Hello, can anybody help here..? I have a list of players and I what to delete an element, but every for loop the list is reverting to ori...
New
senggen
Erlang/OTP 25 [erts-13.2.2] [source] [64-bit] [smp:8:8] [ds:8:8:10] [async-threads:1] 15:22:35.803 [error] gen_event {lager_file_backend...
New
mgjohns61585
Could someone help me? I’m making my first elixir program, number guessing game. I can’t figure out how to convert the user’s guess from ...
New
jerry
Good day to you all. I have been struggling to get a query involving like and ilike to work. Can anyone assist me on this, please? pro...
New
JulienCorb
I am trying to implement my new.html.eex file to create new posts on my website. new.html.eex: <h1>Create Post</h1> <%= ...
New
jaysoifer
Is there a way to rollback a specific migration and only that one (“skipping” all the other ones)? Would mix ecto.rollback -v 200809061...
New
jay1
Why is it that the mnesia database isn’t the most preferred database for use in Elixir/Phoenix?
New
lucidguppy
I have a super simple question about elixir - how would I take a file like this foo bar baz and output a new file that enumerates th...
New
dblack
I’ve got an issue with an app and I’ve no idea of how to troubleshoot it. I’m hoping someone here might have seen something similar. I p...
New
nsuchy
Hi. I’ve noticed that Windows Powershell has it’s own IEX command and you cannot access Elixir’s IEX due to the conflict. This isn’t a cr...
New

Other popular topics Top

vertexbuffer
Hello, can anybody help here..? I have a list of players and I what to delete an element, but every for loop the list is reverting to ori...
New
9mm
I am constructing a JSON object (map) and I need to conditionally set a field. I’m trying to write proper elixir-way code… and I’m at a l...
New
lastday4you
I wanted to check elixir version in phoenix because i found that my elixir is 1.5 but when i use Enum.chunk_by it said the function is un...
New
shahryarjb
Hello, I have map which I want to convert it to string like this: the map: %{last_name: "tavakkoli", name: "shahryar"} the string I ne...
New
dokuzbir
I want to highlight html closing tags when i click a html tag. That works in .html files but doesnt work for html.eex templates. How can...
New
gausby
I asked this very same question on twitter and got some interesting feedback, but I thought it would be a good question to ask here as we...
1207 39297 209
New
dblack
I’ve got an issue with an app and I’ve no idea of how to troubleshoot it. I’m hoping someone here might have seen something similar. I p...
New
hariharasudhan94
I would like to know what is the best IDE for elixir development?
New
PeterCarter
There are pre-rolled solutions for other frameworks that do work. However, Phoenix does not seem to have these. Have people had good expe...
New
hariharasudhan94
Lets say I have map like this fetching from my database %{"_id" => #BSON.ObjectId<58eb1a7a9ad169198c3dXXXX>, "email" => ...
New

We're in Beta

About us Mission Statement