llama

llama

PubSub, too many subscriptions?

I’m a little hazy on what the best practice is for designing a scalable PubSub system. Should there be a few subscriptions, or even just one subscription, from which relevant information is filtered? Or should there be possibly hundreds or more subscriptions which contain very specific information and thus needs less filtering?

Many specific subscriptions seems like the way to go, but at the same time, having users subscribe to hundreds of topics when they log in also seems like it’d slow things down.

Marked As Solved

ruslandoga

ruslandoga

Note that you can add metadata to the subscriptions that can later be used for filtering during dispatch.

Phoenix.PubSub.subscribe(YourApp.PubSub, "topic:user-id", metadata: [some: :extra_info])

which (if you are familiar with Registry) is similar to

Registry.subscribe(YourApp.PubSub, "topic:user-id", [some: :extra_info])

In general, the number of subscriptions doesn’t affect much, as the total throughput should stay about the same:

bench.exs

Mix.install([:benchee, :phoenix_pubsub])

{:ok, _pid} = Phoenix.PubSub.Supervisor.start_link(name: App.PubSub)

defmodule Sub do
  use GenServer

  def start_link(opts \\ []) do
    GenServer.start_link(__MODULE__, opts)
  end

  @impl true
  def init(opts) do
    topic = Keyword.fetch!(opts, :topic)
    Phoenix.PubSub.subscribe(App.PubSub, topic)
    {:ok, nil}
  end

  @impl true
  def handle_info(_message, state) do
    {:noreply, state}
  end
end

Enum.each(1..200_000, fn _ -> Sub.start_link(topic: "topic") end)
Enum.each(1..10, fn _ -> Sub.start_link(topic: "topic2") end)

map = %{"some" => "key", "then" => "some oter key"}

Benchee.run(
  %{
    "message" => fn topic -> Phoenix.PubSub.broadcast(App.PubSub, topic, "message") end,
    "map" => fn topic -> Phoenix.PubSub.broadcast(App.PubSub, topic, map) end
  },
  inputs: %{"lots subs" => "topic", "few subs" => "topic2"}
)

> elixir bench.exs

Operating System: macOS
CPU Information: Apple M1
Number of Available Cores: 8
Available memory: 8 GB
Elixir 1.13.1
Erlang 24.2

Benchmark suite executing with the following configuration:
warmup: 2 s
time: 5 s
memory time: 0 ns
parallel: 1
inputs: few subs, lots subs
Estimated total run time: 28 s

Benchmarking map with input few subs...
Benchmarking map with input lots subs...
Benchmarking message with input few subs...
Benchmarking message with input lots subs...

##### With input few subs #####
Name              ips        average  deviation         median         99th %
map          147.35 K        6.79 μs  ±1531.35%        4.99 μs       24.99 μs
message       95.00 K       10.53 μs  ±9207.44%        2.99 μs        8.99 μs

Comparison:
map          147.35 K
message       95.00 K - 1.55x slower +3.74 μs

##### With input lots subs #####
Name              ips        average  deviation         median         99th %
map              3.12      320.16 ms     ±9.22%      313.18 ms      387.97 ms
message          2.78      360.19 ms    ±16.91%      352.94 ms      479.31 ms

Comparison:
map              3.12
message          2.78 - 1.13x slower +40.02 ms

Where Next?

Popular in Questions Top

chokchit
** (DBConnection.ConnectionError) connection not available and request was dropped from queue after 2733ms. You can configure how long re...
New
vertexbuffer
Hello, can anybody help here..? I have a list of players and I what to delete an element, but every for loop the list is reverting to ori...
New
Harrisonl
We have an ECS cluster with 4 services, where each task joins a single cluster, via discovery ECS discovery service. Currently when I de...
New
Kurisu
For example for a current url like http://localhost:4000/cosmetic/products?_utf8=✓&query=perfume&page=2, I would like to get: ...
New
lessless
I believe there are people here who are dealing with CSV files import on the daily basis, and since Excel is a really popular tool there ...
New
tduccuong
Hi, is there any work on GUI with Elixir, that is similar to Electron/Javascript? My idea is to bundle Phoenix and BEAM into a single se...
New
fireproofsocks
Forgive me if this is obvious, but how does one delete a database record WITHOUT selecting it first? Ecto.Repo — Ecto v3.14.0 has exampl...
New
ashish173
I am using Ecto timestamps with postgres, I can see the timestamps() use the :naive_dateime but for my use case I wanted to store the ti...
New
dotdotdotPaul
Okay, I’m having a heck of a time trying to figure out how to best handle the validation of belongs_to associations in Ecto. I’m sure I’...
New
marick
I had some trouble figuring out how to make many-to-many associations work. Once I got it working, I wrote a blog post. Because I’m a nov...
New

Other popular topics Top

skosch
To my knowledge, put_in, Map.update etc. all have the one limitation of not automatically creating intermediate keys when needed (for exa...
New
Nvim
Anybody knows a comprehensive comparison of Django and Phoenix, thanks for the help. Where are they similar? Where do they differ the m...
New
Patoshizzle
After calling mix ecto.create I get this error: 17:00:32.162 [error] GenServer #PID<0.412.0> terminating ** (Postgrex.Error) FATAL...
New
hariharasudhan94
lets say i have a sample like a = 20; b = 10; if (a > b) do {:ok, "a"} end if (a < b) do {:ok, b} end if (a == b) do {:ok, "equa...
New
pmjoe
I have a relationship of love and hate with Elixir. Lots of things are just absolutely right, but there are some things that are kind of ...
New
SoCreat
i’m a new one to elixir which editor can i use vs code? or atom? Thanks! :smiley:
New
belgoros
I’m not a pro in using Regex and can’t figure out why the following behaviour happens, especially if we take into account the difference ...
New
grych
Hi folks, Few months ago I have announced the proof-of-concept of the library to manipulate the browsers DOM objects directly from Elixi...
639 52341 488
New
ashish173
I am using Ecto timestamps with postgres, I can see the timestamps() use the :naive_dateime but for my use case I wanted to store the ti...
New
marick
I had some trouble figuring out how to make many-to-many associations work. Once I got it working, I wrote a blog post. Because I’m a nov...
New

We're in Beta

About us Mission Statement