I have an IOT application which has lots of IoT nodes sending data to a phoenix server. There is a notification tool, where a client is able to set rules based on which, he needs to be notified if, there is some anomaly. To handle this I have created a Manager with a worker pool to evaluate every data point, and send the notification. Since every request is going to be checked, how should I decide on the number of workers I should keep for handling the requests?
I don’t want to use Task as there can be thousands of requests from IoT devices out there and it may result in too many processes which may exceed the capacity.