Is min_demand automatically calculated from max_demand?

Questions / Help
,
1

Hello,

I’m using Flow 0.14.3 with Elixir 1.8.1 and Erlang/OTP 21.2.

Is the min_demand option automatically calculated from max_demand? :thinking: The reason I ask is that I’m specifying max_demand in some of my Flow partitions and I want to know whether I should also specify min_demand every time. :man_shrugging:

The closest I got to answering this question myself was to find this snippet in the GenStage source code which seems to suggest that min_demand is automatically calculated, but I’m uncertain whether this applies to all of GenStage or if at all to Flow.

with {:ok, max, _} <- Utils.validate_integer(opts, :max_demand, 1000, 1, :infinity, false),
     {:ok, min, _} <- Utils.validate_integer(opts, :min_demand, div(max, 2), 0, max - 1, false),

Could someone knowledgeable in Flow/GenStage please confirm?

Thanks for your consideration.

4 Likes
2

Confirm.

3 Likes
3

As @josevalim confirmed, it is calculated. The min demand is calculated as half the max demand.

What I noticed when building a job queue was that with a mixture of fast and slow jobs the max demand was rarely met. Overriding the min demand with a low number, i.e. 1, will force it to get closer to the max demand but with the drawback that there is a lot more dispatching.