Hi there,

I have a queue with the following opts:

{"paused": false, "ack_async": null, "rate_limit": null, "local_limit": 100, "global_limit": {"allowed": 1, "partition": {"keys": ["our_key"], "fields": ["args"]}}, "retry_backoff": 1000, "retry_attempts": 5, "refresh_interval": null}

I have thousands of jobs in this queue, with 7 different our_key, so I’m expecting to see 7 jobs running concurrently in my cluster, however, only 5 are running.

I have two producers, with the following meta:

{"paused": false, "rate_limit": null, "local_limit": 100, "global_limit": {"allowed": 1, "tracked": {"27783521": {"args": {"our_key": "SW5zdGFuY2U6YjU5MWI4ZmItYjk1OS00ODkyLTgzZTEtMmQ3NmVmMjhjMDc2"}, "count": 1, "worker": null}, "126742740": {"args": {"our_key": "SW5zdGFuY2U6MjI1NzI2MzgtZDg0ZC00MDYwLWIwZDItMDQ0MDNiZTc5ODFm"}, "count": 1, "worker": null}}, "partition": {"keys": ["our_key"], "fields": ["args"]}}, "retry_backoff": 1000, "retry_attempts": 5}

{"paused": false, "rate_limit": null, "local_limit": 100, "global_limit": {"allowed": 1, "tracked": {"73912501": {"args": {"our_key": "SW5zdGFuY2U6YmYyY2IzZDYtMjhmMS00ZGRjLWE5NDEtOTU4OGM3Y2Y1YTFl"}, "count": 1, "worker": null}, "133456980": {"args": {"our_key": "SW5zdGFuY2U6NDg3MzFhN2ItOGRjNC00ZTM2LWFlMzgtOTM2N2Q1M2E4MDkx"}, "count": 1, "worker": null}, "133940658": {"args": {"our_key": "SW5zdGFuY2U6NzA5NzE1ZDEtYWM4OS00ZTgyLTg3N2UtZDhhZGU5NGE5NWZk"}, "count": 1, "worker": null}}, "partition": {"keys": ["our_key"], "fields": ["args"]}}, "retry_backoff": 1000, "retry_attempts": 5}

How can I know what’s preventing the other 2 jobs to start?

The oban jobs table aggregation by our key:

select args->>'our_key', count(*) from oban_jobs where queue = 'our_queue' group by 1;

SW5zdGFuY2U6YjU5MWI4ZmItYjk1OS00ODkyLTgzZTEtMmQ3NmVmMjhjMDc2	8316
SW5zdGFuY2U6ODQ4ZDUyODktYjcxMy00Mjg1LTg5YWItMzJiNjgwYjI4ODZl	322
SW5zdGFuY2U6YmYyY2IzZDYtMjhmMS00ZGRjLWE5NDEtOTU4OGM3Y2Y1YTFl	1172
SW5zdGFuY2U6NDg3MzFhN2ItOGRjNC00ZTM2LWFlMzgtOTM2N2Q1M2E4MDkx	713
SW5zdGFuY2U6ZWM1NzA0Y2YtNDk3MC00YzI3LWE5ZjgtYmY0ZTY1YjJlYzJi	401
SW5zdGFuY2U6NzA5NzE1ZDEtYWM4OS00ZTgyLTg3N2UtZDhhZGU5NGE5NWZk	1951
SW5zdGFuY2U6MjI1NzI2MzgtZDg0ZC00MDYwLWIwZDItMDQ0MDNiZTc5ODFm	255

The queue is running only the 5 jobs, so it’s not the local limit.

Oban versions:

"oban": {:hex, :oban, "2.18.3"
"oban_met": {:hex, :oban_met, "0.1.7"
"oban_pro": {:hex, :oban_pro, "1.4.13"
"oban_web": {:hex, :oban_web, "2.10.5"

Oban config:

engine: Oban.Pro.Engines.Smart,
notifier: Oban.Notifiers.PG,