Number of Workers in a Pool for IOT notification APP

I have an IOT application which has lots of IoT nodes sending data to a phoenix server. There is a notification tool, where a client is able to set rules based on which, he needs to be notified if, there is some anomaly. To handle this I have created a Manager with a worker pool to evaluate every data point, and send the notification. Since every request is going to be checked, how should I decide on the number of workers I should keep for handling the requests?
I don’t want to use Task as there can be thousands of requests from IoT devices out there and it may result in too many processes which may exceed the capacity.

1 Like

Honestly this sounds like the levels of genstage/flow/broadway?

Hi, @OvermindDL1 can you please guide me towards some resources to understand the terms you have mentioned, if possible? Thanks in advance!

1 Like

All of them are made by the authors of Elixir itself, they are different ‘levels’ of essentially staged processing.

Broadway (most specific):

Flow (flow based staged processing):

GenStage (the stages themselves, low level):