I have been searching around, but am still in need of a clear explanation of how min_demand and max_demand work with Broadway.
In our use case, we have a low volume of messages coming in from rabbitMQ, but we need to process the messages as fast as possible when they come in. Let’s say we have 1 message per second coming in, and it takes 0.5 seconds to process a message.
What happens with the default values of min_demand (500) and max_demand (1000)?
The processor will request 500 initially, and get 1. Will it wait for 499 more messages before doing anything, or will it process that first message right away?
Basically I want to know - what values should I set to ensure that messages are not left waiting before being processed?
Assuming it works like GenStage, the first request will be for max_demand, and then once max_demand - min_demand has been filled, it will request max_demand - min_demand.
It will being processing as soon as it gets anything, even if it is less than max_demand - min_demand. So in your scenario it would request 1000, get 1 and it would start processing the single message immediately.
Note that the default values for min and max demand in Broadway is 5 and 10, not 500 and 1000.
