zampino

zampino

Selective Broadcasts

This proposal probably arises from my immature experience with GenStage,
but what if, at subscribe time a consumer could specify a selector i.e. a function
(event :: any -> boolean) which would filter events being sent to itself. Somewhat like:

GenStage.sync_subscribe(consumer, 
  to: producer, 
  selector: fn %{key: key} -> key =~ "foo" end
)

Now when producer emits an event say %{key: key}, this will be dispatched to consumer
only if "foo" matches key.
This is probably a similar approach to the partition dispatcher with extra control over the hashing function. Could I state this so?

I have a very naïve proof of concept in the form of a private pull request here.

where I also modified the gen_event example:

What would the community think of this feature?
Is there already a way to achieve this with no need of extra work?

Most Liked

benwilson512

benwilson512

Author of Craft GraphQL APIs in Elixir with Absinthe

This seems to be essentially what Partition Dispatcher does: https://hexdocs.pm/gen_stage/Experimental.GenStage.PartitionDispatcher.html#content

zampino

zampino

…to give some further motivations,
we’re working with a dynamic population of stages which all subscribe to some producer
with broadcast dispatch.
Consumer stages gets created and updated continuously based on a stream of user events.

Each node might observe some other for changes derived from receiving
events from the source.
This is the reason why I’d need to specify which events I might be interested in, assuming that such events, carry the identifier of the node to which they’re primarily addressed.


At this point a question arises, what if I’d need to update the subscription options
(not necessarily the :selector, but in general)
at present dispatchers save they’re consumers (demands) in a list:

https://github.com/elixir-lang/gen_stage/blob/accec040079e93cbc1b9a57f0d22a365bcb4771e/lib/gen_stage/broadcast_dispatcher.ex#L82-L84

and not in a map (by ref), what is the intention behind this behaviour, maybe @josevalim?
Can I update subscription options without canceling and subscribing again?
If I’m not wrong, if I subscribe twice to a producer, I will receive events twice, is it correct?

zampino

zampino

I agree, especially in the demand auto-adjustment,
that could cause problems.

If I wish to change the selector then,
one could first re-subscribe with a new selector
and subsequently cancel the old ref, in this case a consumer
might get at most double messages
until the dispatcher cancels the old demand.

Would you suggest to do so?

Better guarding against duplicates in the stage,
than possibly losing events by first canceling and then re-subscribing right?

Where Next?

Popular in Discussions Top

sashaafm
I’m trying to evaluate the best combo/stack for a BEAM Web app. Right now I’m exploring Yaws a bit, after having dealt with Phoenix for a...
New
thojanssens1
It would be nice to be able to define a redirect from one route to another from the router.ex file. E.g.: redirect "/", UserController, ...
New
Fl4m3Ph03n1x
Background A few days ago I was listening to The future of Elixir from Elixir Talks, with Dave Thomas (@pragdave ) and Brian Mitchell. I...
New
nunobernardes99
Hi there Elixir friends :vulcan_salute: In a recent task I was on, I needed to check in two dates which of them is the maximum and which...
New
marciol
Please, let me know if this kind of discussion already took place in another topic . Hi all, how do you consider if is better to build ...
New
PragTob
Hey everyone, this has been on my mind for some time and I’d love your input on it! TLDR: I feel like maps are superioer for storing and...
New
rms.mrcs
A couple of days ago I was discussing with a friend about different approaches to write microservices. He said that if he was going to w...
New
jsonify
So, is Heroku the only free option for hosting Phoenix/Elixir at this point? I’m not ready to commit to paying monthly and was wondering ...
New
klo
Got a question about when to concat vs. prepending items to list then reversing to achieve appending. So i know lists boil down to [1 | ...
New
dogweather
I wrote this comment on r/haskell, and it’s not popular there. :wink: But I think I’m on to something… Haskell reminds me of Java, and e...
New

Other popular topics Top

Harrisonl
We have an ECS cluster with 4 services, where each task joins a single cluster, via discovery ECS discovery service. Currently when I de...
New
Fl4m3Ph03n1x
About me? ( if you have nothing better to do than reading about some random guy in the internet :stuck_out_tongue: ) Hello all, this is ...
New
msaraiva
Surface is an experimental library built on top of Phoenix LiveView and its new LiveComponent API that aims to provide a more declarative...
564 43591 214
New
stefanchrobot
What’s the safe way to decode a JSON string into a struct? I want to avoid calling String.to_atom. Jason.decode can give me a map with st...
New
josevalim
Hi everyone, One of the features added to Elixir early on to help integration with Erlang code was the idea of overridable function defi...
New
JeremM34
Hello, how can I check the Phoenix version ? Thanks !
New
nsuchy
Hi. I’ve noticed that Windows Powershell has it’s own IEX command and you cannot access Elixir’s IEX due to the conflict. This isn’t a cr...
New
Brian
What is the proper way to load a module from a file in to IEX? In the python world, doing something like this pretty standard: from ....
New
WestKeys
Currently suffering from paralysis by [HTTP client] analysis. This is rather unusual in Elixirland as there tends to be consensus on the ...
New
hariharasudhan94
Lets say i have map like this fetching from my database %{"_id" => #BSON.ObjectId<58eb1a7a9ad169198c3dXXXX>, "email" => "XX...
New

We're in Beta

About us Mission Statement