Can't start application locally - failed to start child: Oban

I have an application which has always worked fine locally, and works well in production but today I’ve been trying to start it iex -S mix phx.server and I get this error message:

GenServer {Oban.Registry, {Oban, {:plugin, Oban.Plugins.Cron}}} terminating
** (DBConnection.ConnectionError) connection not available and request was dropped from queue after 2964ms. This means requests are coming in and your connection pool cannot serve them fast enough. You can address this by:

  1. Ensuring your database is available and that you can connect to it
  2. Tracking down slow queries and making sure they are running fast enough
  3. Increasing the pool_size (although this increases resource consumption)
  4. Allowing requests to wait longer by increasing :queue_target and :queue_interval

See DBConnection.start_link/2 for more information
...

Application scrivener exited: :stopped
** (Mix) Could not start application myapp_data: MyApp.Application.start(:normal, []) returned an error: shutdown: failed to start child: Oban
    ** (EXIT) shutdown: failed to start child: "accounting"
        ** (EXIT) shutdown: failed to start child: Oban.Queue.Producer
            ** (EXIT) exited in: GenServer.call(#PID<0.1122.0>, {:listen, [:insert, :signal]}, 5000)
                ** (EXIT) time out

I’m on the main branch, no changes. I don’t have any other beam instances running and just to be sure I ran killall -9 beam.smp. I’ve removed _build no luck.

I’ve had similar issues in the past but never this persistent and generally one of the above solves the issue.

Just looks like Postgres stopped running. If you type psql on the command line, does it work?

Wow you’re fast! Postgres is running, I can view my databases with psql or through my SQL client.

I should mention that other Elixir apps start without issue.

lol, ya I’m having a bit of a day today and spending a lot of time on this forum.

I’m not sure as I don’t really have experience with Oban. Just reading the error, though, is/was there something shovelling a crapton of jobs onto the queue and it can’t handle it? Maybe try truncating the queue table? Otherwise, sorry for the noise, I’m not really sure :frowning:

1 Like

Something is monopolizing all of your db connections and Oban isn’t able to make the necessary requests on startup. Cron only runs at the top of every minute, so it’s odd that it would affect booting.

You could try increasing the pool size, or don’t start Oban on application boot temporarily and see if the system can normalize.

That’s doubtful. Inserting jobs happens from application processes, not Oban’s, and an exception while fetching jobs would be from a Producer rather than Cron.

1 Like

Apologies everyone it appears my dev.secrets.exs file had the wrong DB creds and was trying to connect to the production DB rather than my local instance! Thanks @sodapopcan and @sorentwo for your quick responses.

1 Like