Hello, we recently upgraded a few services:

• elixir 1.15.8-otp-25 to 1.18.4-otp-27 (live deployed)
• oban ~> 2.18.1 to ~> 2.19
• oban_pro ~> 1.4.11 to ~> 1.5.0
• oban_web ~> 2.10.2 to ~> 2.11

In my current task, I’m trying to upgrade 3 oban packages and it works fine locally. However, when starting application in a remote (testing) environment, it failed to start.

I have reverted oban versions and the app starts normally in test env. Remove cache in test env doesn’t help. We have postgresql 14.17

Any help on what’s happening? Please let me know if you need anything @sorentwo @sorenone . Thanks!

This error is logged in the remote server.

{"time":"2025-09-16T11:24:52.614Z","severity":"error",
"message":"GenServer {Oban.Registry, {Oban, {:producer, \"migration\"}}} terminating\n** (ArgumentError) errors were found at the given arguments:
 * 1st argument: the table identifier does not refer to an existing ETS table 
 (stdlib 7.0.2) :ets.select(:pro_ack_tab_0, [{{{:ack, \"Oban\", \"migration\", :_}, :_, :_, :_}, [], [:\"$_\"]}])
 (oban_pro 1.5.5) lib/oban/pro/engines/smart.ex:1012: Oban.Pro.Engines.Smart.get_acks/1
 (oban_pro 1.5.5) lib/oban/pro/engines/smart.ex:552: Oban.Pro.Engines.Smart.fetch_jobs/3
 (oban 2.19.4) lib/oban/engine.ex:252: anonymous fn/4 in Oban.Engine.fetch_jobs/3
 (oban 2.19.4) lib/oban/engine.ex:387: anonymous fn/3 in Oban.Engine.with_span/4
 (telemetry 1.3.0) /__w/balanced/balanced/deps/telemetry/src/telemetry.erl:324: :telemetry.span/3
 (oban 2.19.4) lib/oban/queue/producer.ex:253: Oban.Queue.Producer.start_jobs/1
 (oban 2.19.4) lib/oban/queue/producer.ex:244: anonymous fn/2 in Oban.Queue.Producer.dispatch/1 Last message: :dispatch
 State: %Oban.Queue.Producer{conf: %Oban.Config{dispatch_cooldown: 250, engine: Oban.Pro.Engines.Smart, get_dynamic_repo: nil, insert_trigger: true, log: false, name: Oban, node: \"pr-4147-84588b87fc-cbwqg\", notifier: {Oban.Notifiers.Postgres, []}, peer: {Oban.Peers.Database, []}, plugins: [{Oban.Pro.Plugins.DynamicPruner, [state_overrides: [cancelled: {:max_age, {1, :hour}}, completed: {:max_age, {1, :month}}, discarded: {:max_age, {3, :month}}]]}, {Oban.Pro.Plugins.DynamicLifeline, []}], prefix: \"public\", queues: [...truncated...], repo: Balanced.Repo, shutdown_grace_period: 15000, stage_interval: 1000, testing: :disabled}, foreman: {:via, Registry, {Oban.Registry, {Oban, {:foreman, \"migration\"}}}}, meta: %Oban.Pro.Producer{__meta__: #Ecto.Schema.Metadata<:loaded, \"public\", \"oban_producers\">, uuid: \"01995245-2893-7c51-af4c-60a4400ca0a4\", name: \"Oban\", node: \"pr-4147-84588b87fc-cbwqg\", queue: \"migration\", started_at: ~U[2025-09-16 11:24:48.147420Z], updated_at: ~U[2025-09-16 11:24:48.147424Z], ack_async: true, ack_tab: :pro_ack_tab_0, refresh_interval: 30000, xact_delay: 1000, xact_retry: 5, xact_timeout: 30000, meta: %Oban.Pro.Producer.Meta{local_limit: 1, paused: false, shutdown_started_at: nil, global_limit: nil, rate_limit: nil}}, name: {:via, Registry, {Oban.Registry, {Oban, {:producer, \"migration\"}}}}, dispatch_timer: #Reference<0.534427290.921698305.255298>, refresh_timer: #Reference<0.534427290.921698305.253530>, dispatch_cooldown: 250, running: %{}}",
 "metadata":{"error":{"initial_call":null,"reason":"** (ArgumentError) errors were found at the given arguments:\n\n  * 1st argument: the table identifier does not refer to an existing ETS table\n\n    (stdlib 7.0.2) :ets.select(:pro_ack_tab_0, [{{{:ack, \"Oban\", \"migration\", :_}, :_, :_, :_}, [], [:\"$_\"]}])\n    (oban_pro 1.5.5) lib/oban/pro/engines/smart.ex:1012: Oban.Pro.Engines.Smart.get_acks/1\n    (oban_pro 1.5.5) lib/oban/pro/engines/smart.ex:552: Oban.Pro.Engines.Smart.fetch_jobs/3\n    (oban 2.19.4) lib/oban/engine.ex:252: anonymous fn/4 in Oban.Engine.fetch_jobs/3\n    (oban 2.19.4) lib/oban/engine.ex:387: anonymous fn/3 in Oban.Engine.with_span/4\n    (telemetry 1.3.0) /__w/balanced/balanced/deps/telemetry/src/telemetry.erl:324: :telemetry.span/3\n    (oban 2.19.4) lib/oban/queue/producer.ex:253: Oban.Queue.Producer.start_jobs/1\n    (oban 2.19.4) lib/oban/queue/producer.ex:244: anonymous fn/2 in Oban.Queue.Producer.dispatch/1\n"},"error_logger":{"tag":"error","report_cb":"&:gen_server.format_log/1"},"function":"error_info/7","line":2785,"module":"gen_server","time":1758021892614425,"file":"gen_server.erl","domain":"[:otp]","erl_level":"error"}}

▸  Evaluation failed with: errors were found at the given arguments:
▸    * 1st argument: the table identifier does not refer to an existing ETS table

Our config looks like

config :myapp, Oban,
  repo: Myapp.Repo,
  engine: Oban.Pro.Engines.Smart,
  plugins: [
    Oban.Pro.Plugins.DynamicLifeline,
    {
      Oban.Pro.Plugins.DynamicPruner,
      state_overrides: [
        cancelled: {:max_age, {1, :hour}},
        completed: {:max_age, {1, :month}},
        discarded: {:max_age, {3, :month}}
      ]
    }
  ],
  queues: [
    default: 1,
    other_jobs: 1,
    ... truncated
  ],
  dispatch_cooldown: 250

And all children are started in application file

  def start(_type, _args) do
    maybe_install_ecto_dev_logger()

    Appsignal.Phoenix.LiveView.attach()

    children =
      maybe_cluster_supervisor() ++
        cache_supervisors() ++
        [
          MyAppWeb.Endpoint,
          MyApp.PromEx,
          MyApp.Repo,
          MyApp.Vault,
          MyAppWeb.Telemetry,
          {Phoenix.PubSub, [name: MyApp.PubSub]},
          {Task.Supervisor, name: MyApp.TaskSupervisor},
          {Oban, oban_config()}
        ]

    Oban.Telemetry.attach_default_logger()
    Oban.Web.Telemetry.attach_default_logger()

    opts = [strategy: :one_for_one, name: MyApp.Supervisor]
    Supervisor.start_link(children, opts)
  end

  defp oban_config do
    config = Application.fetch_env!(:myapp, Oban)

    if config[:plugins] do
      Keyword.update!(config, :plugins, fn plugins ->
        Keyword.put(plugins, Oban.Pro.Plugins.DynamicCron, crontab: periodical_jobs(), timezone: "Etc/UTC")
      end)
    else
      config
    end
  end