Hello, we recently upgraded a few services:
- elixir 1.15.8-otp-25 to 1.18.4-otp-27 (live deployed)
- oban ~> 2.18.1 to ~> 2.19
- oban_pro ~> 1.4.11 to ~> 1.5.0
- oban_web ~> 2.10.2 to ~> 2.11
In my current task, I’m trying to upgrade 3 oban packages and it works fine locally. However, when starting application in a remote (testing) environment, it failed to start.
I have reverted oban versions and the app starts normally in test env. Remove cache in test env doesn’t help. We have postgresql 14.17
Any help on what’s happening? Please let me know if you need anything @sorentwo @sorenone . Thanks!
This error is logged in the remote server.
{"time":"2025-09-16T11:24:52.614Z","severity":"error",
"message":"GenServer {Oban.Registry, {Oban, {:producer, \"migration\"}}} terminating\n** (ArgumentError) errors were found at the given arguments:
* 1st argument: the table identifier does not refer to an existing ETS table
(stdlib 7.0.2) :ets.select(:pro_ack_tab_0, [{{{:ack, \"Oban\", \"migration\", :_}, :_, :_, :_}, [], [:\"$_\"]}])
(oban_pro 1.5.5) lib/oban/pro/engines/smart.ex:1012: Oban.Pro.Engines.Smart.get_acks/1
(oban_pro 1.5.5) lib/oban/pro/engines/smart.ex:552: Oban.Pro.Engines.Smart.fetch_jobs/3
(oban 2.19.4) lib/oban/engine.ex:252: anonymous fn/4 in Oban.Engine.fetch_jobs/3
(oban 2.19.4) lib/oban/engine.ex:387: anonymous fn/3 in Oban.Engine.with_span/4
(telemetry 1.3.0) /__w/balanced/balanced/deps/telemetry/src/telemetry.erl:324: :telemetry.span/3
(oban 2.19.4) lib/oban/queue/producer.ex:253: Oban.Queue.Producer.start_jobs/1
(oban 2.19.4) lib/oban/queue/producer.ex:244: anonymous fn/2 in Oban.Queue.Producer.dispatch/1 Last message: :dispatch
State: %Oban.Queue.Producer{conf: %Oban.Config{dispatch_cooldown: 250, engine: Oban.Pro.Engines.Smart, get_dynamic_repo: nil, insert_trigger: true, log: false, name: Oban, node: \"pr-4147-84588b87fc-cbwqg\", notifier: {Oban.Notifiers.Postgres, []}, peer: {Oban.Peers.Database, []}, plugins: [{Oban.Pro.Plugins.DynamicPruner, [state_overrides: [cancelled: {:max_age, {1, :hour}}, completed: {:max_age, {1, :month}}, discarded: {:max_age, {3, :month}}]]}, {Oban.Pro.Plugins.DynamicLifeline, []}], prefix: \"public\", queues: [...truncated...], repo: Balanced.Repo, shutdown_grace_period: 15000, stage_interval: 1000, testing: :disabled}, foreman: {:via, Registry, {Oban.Registry, {Oban, {:foreman, \"migration\"}}}}, meta: %Oban.Pro.Producer{__meta__: #Ecto.Schema.Metadata<:loaded, \"public\", \"oban_producers\">, uuid: \"01995245-2893-7c51-af4c-60a4400ca0a4\", name: \"Oban\", node: \"pr-4147-84588b87fc-cbwqg\", queue: \"migration\", started_at: ~U[2025-09-16 11:24:48.147420Z], updated_at: ~U[2025-09-16 11:24:48.147424Z], ack_async: true, ack_tab: :pro_ack_tab_0, refresh_interval: 30000, xact_delay: 1000, xact_retry: 5, xact_timeout: 30000, meta: %Oban.Pro.Producer.Meta{local_limit: 1, paused: false, shutdown_started_at: nil, global_limit: nil, rate_limit: nil}}, name: {:via, Registry, {Oban.Registry, {Oban, {:producer, \"migration\"}}}}, dispatch_timer: #Reference<0.534427290.921698305.255298>, refresh_timer: #Reference<0.534427290.921698305.253530>, dispatch_cooldown: 250, running: %{}}",
"metadata":{"error":{"initial_call":null,"reason":"** (ArgumentError) errors were found at the given arguments:\n\n * 1st argument: the table identifier does not refer to an existing ETS table\n\n (stdlib 7.0.2) :ets.select(:pro_ack_tab_0, [{{{:ack, \"Oban\", \"migration\", :_}, :_, :_, :_}, [], [:\"$_\"]}])\n (oban_pro 1.5.5) lib/oban/pro/engines/smart.ex:1012: Oban.Pro.Engines.Smart.get_acks/1\n (oban_pro 1.5.5) lib/oban/pro/engines/smart.ex:552: Oban.Pro.Engines.Smart.fetch_jobs/3\n (oban 2.19.4) lib/oban/engine.ex:252: anonymous fn/4 in Oban.Engine.fetch_jobs/3\n (oban 2.19.4) lib/oban/engine.ex:387: anonymous fn/3 in Oban.Engine.with_span/4\n (telemetry 1.3.0) /__w/balanced/balanced/deps/telemetry/src/telemetry.erl:324: :telemetry.span/3\n (oban 2.19.4) lib/oban/queue/producer.ex:253: Oban.Queue.Producer.start_jobs/1\n (oban 2.19.4) lib/oban/queue/producer.ex:244: anonymous fn/2 in Oban.Queue.Producer.dispatch/1\n"},"error_logger":{"tag":"error","report_cb":"&:gen_server.format_log/1"},"function":"error_info/7","line":2785,"module":"gen_server","time":1758021892614425,"file":"gen_server.erl","domain":"[:otp]","erl_level":"error"}}
▸ Evaluation failed with: errors were found at the given arguments:
▸ * 1st argument: the table identifier does not refer to an existing ETS table
Our config looks like
config :myapp, Oban,
repo: Myapp.Repo,
engine: Oban.Pro.Engines.Smart,
plugins: [
Oban.Pro.Plugins.DynamicLifeline,
{
Oban.Pro.Plugins.DynamicPruner,
state_overrides: [
cancelled: {:max_age, {1, :hour}},
completed: {:max_age, {1, :month}},
discarded: {:max_age, {3, :month}}
]
}
],
queues: [
default: 1,
other_jobs: 1,
... truncated
],
dispatch_cooldown: 250
And all children are started in application file
def start(_type, _args) do
maybe_install_ecto_dev_logger()
Appsignal.Phoenix.LiveView.attach()
children =
maybe_cluster_supervisor() ++
cache_supervisors() ++
[
MyAppWeb.Endpoint,
MyApp.PromEx,
MyApp.Repo,
MyApp.Vault,
MyAppWeb.Telemetry,
{Phoenix.PubSub, [name: MyApp.PubSub]},
{Task.Supervisor, name: MyApp.TaskSupervisor},
{Oban, oban_config()}
]
Oban.Telemetry.attach_default_logger()
Oban.Web.Telemetry.attach_default_logger()
opts = [strategy: :one_for_one, name: MyApp.Supervisor]
Supervisor.start_link(children, opts)
end
defp oban_config do
config = Application.fetch_env!(:myapp, Oban)
if config[:plugins] do
Keyword.update!(config, :plugins, fn plugins ->
Keyword.put(plugins, Oban.Pro.Plugins.DynamicCron, crontab: periodical_jobs(), timezone: "Etc/UTC")
end)
else
config
end
end
