Hey, recently I started to have issues during deployments to Digital Ocean App Platform. I’m deploying an Elixir release packed as a Docker image. During startup I’m often getting a database connection error:
[myapp] [2023-03-26 09:31:39] 09:31:39.044 [error] Could not create schema migrations table. This error usually happens due to the following:
[myapp] [2023-03-26 09:31:39]
[myapp] [2023-03-26 09:31:39] * The database does not exist
[myapp] [2023-03-26 09:31:39] * The "schema_migrations" table, which Ecto uses for managing
[myapp] [2023-03-26 09:31:39] migrations, was defined by another library
[myapp] [2023-03-26 09:31:39] * There is a deadlock while migrating (such as using concurrent
[myapp] [2023-03-26 09:31:39] indexes with a migration_lock)
[myapp] [2023-03-26 09:31:39]
[myapp] [2023-03-26 09:31:39] To fix the first issue, run "mix ecto.create".
[myapp] [2023-03-26 09:31:39]
[myapp] [2023-03-26 09:31:39] To address the second, you can run "mix ecto.drop" followed by
[myapp] [2023-03-26 09:31:39] "mix ecto.create". Alternatively you may configure Ecto to use
[myapp] [2023-03-26 09:31:39] another table and/or repository for managing migrations:
[myapp] [2023-03-26 09:31:39]
[myapp] [2023-03-26 09:31:39] config :myapp, MyApp.Repo,
[myapp] [2023-03-26 09:31:39] migration_source: "some_other_table_for_schema_migrations",
[myapp] [2023-03-26 09:31:39] migration_repo: AnotherRepoForSchemaMigrations
[myapp] [2023-03-26 09:31:39]
[myapp] [2023-03-26 09:31:39] The full error report is shown below.
[myapp] [2023-03-26 09:31:39]
[myapp] [2023-03-26 09:31:39] ** (DBConnection.ConnectionError) connection not available and request was dropped from queue after 2483ms. This means requests are coming in and your connection pool cannot serve them fast enough. You can address this by:
[myapp] [2023-03-26 09:31:39]
[myapp] [2023-03-26 09:31:39] 1. Ensuring your database is available and that you can connect to it
[myapp] [2023-03-26 09:31:39] 2. Tracking down slow queries and making sure they are running fast enough
[myapp] [2023-03-26 09:31:39] 3. Increasing the pool_size (although this increases resource consumption)
[myapp] [2023-03-26 09:31:39] 4. Allowing requests to wait longer by increasing :queue_target and :queue_interval
[myapp] [2023-03-26 09:31:39]
[myapp] [2023-03-26 09:31:39] See DBConnection.start_link/2 for more information
[myapp] [2023-03-26 09:31:39]
[myapp] [2023-03-26 09:31:39] (ecto_sql 3.9.2) lib/ecto/adapters/sql.ex:913: Ecto.Adapters.SQL.raise_sql_call_error/1
[myapp] [2023-03-26 09:31:39] (elixir 1.14.3) lib/enum.ex:1658: Enum."-map/2-lists^map/1-0-"/2
[myapp] [2023-03-26 09:31:39] (ecto_sql 3.9.2) lib/ecto/adapters/sql.ex:1005: Ecto.Adapters.SQL.execute_ddl/4
[myapp] [2023-03-26 09:31:39] (ecto_sql 3.9.2) lib/ecto/migrator.ex:677: Ecto.Migrator.verbose_schema_migration/3
[myapp] [2023-03-26 09:31:39] (ecto_sql 3.9.2) lib/ecto/migrator.ex:491: Ecto.Migrator.lock_for_migrations/4
[myapp] [2023-03-26 09:31:39] (ecto_sql 3.9.2) lib/ecto/migrator.ex:403: Ecto.Migrator.run/4
[myapp] [2023-03-26 09:31:39] (ecto_sql 3.9.2) lib/ecto/migrator.ex:146: Ecto.Migrator.with_repo/3
[myapp] [2023-03-26 09:31:39] nofile:1: (file)
My migration task seems unable to connect to the database.
- The DB exists and is reachable via other means,
- The DB connection pool size is 22,
- The app’s pool size is set to 6,
- I’m only running one instance,
- I use zero-downtime deployments, so max connections should be 12 during deployment.
I’m only experiencing this during deployments which makes them fail. Any idea what might be causing this?
Not sure what is going on here. It feels like it’s working sometimes. Sounds like some sort of a race condition.
