I’m using Oban PRO and I’m facing a job distribution issue.
When executing some queries on the database, I noticed that some nodes were executing much more jobs than others. For example, during one day, there was a node that executed 1M jobs, while another executed only 100k. This caused a CPU and memory increase in nodes was running more jobs.

My configs:

• Oban 2.17.12
• Oban Pro 1.4.10
• Queue :foo with limit: 20, EngineSmart, partitioned and plugins: Reindexer, DynamicLifeline and DynamicPartitioner
• 10 nodes (pods) in Kubernetes
• Some jobs are running in priority 0 and others in priority 1 or 2, depending on the args in the job

Here is my complete config:

peer: Oban.Peers.Postgres,
repo: MyRepo,
notifier: Oban.Notifiers.PG,
shutdown_grace_period: my_timeout,
stage_interval: :timer.seconds(2),
prefix: "partitioned_oban",
engine: Oban.Pro.Engines.Smart,
plugins: [
   {Oban.Plugins.Reindexer, schedule: my_schedule, timezone: my_timezone, timeout: :timer.minutes(2)},
   {Oban.Pro.Plugins.DynamicLifeline, rescue_interval: :timer.minutes(5)},
   {
	  Oban.Pro.Plugins.DynamicPartitioner,
	  buffer: 2,
	  schedule: my_schedule,
	  timezone: my_timezone,
	  retention: [completed: 1, cancelled: 1, discarded: 30]
   }
],
queues: [
  foo_jobs: 20,
  rate_limited_foo_jobs: [
    local_limit: 20,
    rate_limit: [
      allowed: 10,
      period: 60
    ]
  ]
]