gpr
keys distribution between processes
Hi. I have the following task:
I have a server, which support subscription to different topics. One connection supports subscription for up to some limited number of topics, say 1000. I want to subscribe to, say, up to 10000 topics, so I use pool of up to 10 connections. Connections can fail and be restarted. The task is to distribute topics between connections and I would like to avoid manual tracking of which topic belongs to which connection. If number of subscribed topics is low, I would like to migrate topics between connections and shutdown some connections.
If I abstract from topics/connections, task sounds like distributing keys (topics) between processes with some configuration of limits, migration strategy, etc. I’m doing it manually now, but code is more complex than I wanted it to be, and I’m wondering, if there are some libraries in erlang/elixir which support such key distribution algorithms.
ANy suggestions are welcome.
First Post!
dmitrykleymenov
I’m not completely sure what you mean, but if i’ve understood correctly, consistent hashing is what you need. If so, this may help.
Popular in Questions
Other popular topics
Latest on Elixir Forum
Sponsor Spotlight
Move fast and fix things! Error tracking with less noise for Elixir devs who love to ship. Start your free dev account.
Categories:
Sub Categories:
Forums
Popular Tags
- #ecto
- #liveview
- #troubleshooting
- #learning-elixir
- #deployment
- #library
- #erlang
- #testing
- #genserver
- #mix
- #absinthe
- #remote-other
- #otp
- #plug
- #how-to-question
- #macros
- #postgres
- #channels
- #elixirconf
- #exunit
- #discussion
- #code-sync
- #javascript
- #podcasts
- #onsite
- #dialyzer
- #docker
- #authentication
- #umbrella
- #full-time-contract
- #podcasts-by-brainlid
- #ecto-query
- #elixir-ls
- #phoenix_html
- #iex
- #blog-post
- #graphql
- #genstage
- #ai
- #websockets
- #supervisor
- #elixirconf-us
- #advent-of-code
- #distillery
- #processes
- #api
- #forms
- #metaprogramming
- #security
- #hex
Our Sponsors
Build Elixir applications with speed and confidence.
Supporting innovation across the BEAM ecosystem.
We build reliable cloud platforms for business-critical systems.
Catch errors, track performance, monitor hosts and more.
Error tracking for Elixir devs who love to ship. Start your free account.
Practical resources that improve the lives of professional developers.
Develop your skills with books, videos, and courses.
The deployment platform built for Elixir: PaaS ease, VPS control.
The team behind Membrane, Popcorn, LiveDebugger. Available for hire.
Courses that'll move you from confusion to "Aha, now I get it!"