Postgrex :econnrefused after ecto migration/seed in umbrella app

Using elixir:1.8.1-alpine,(ecto 3.0.9, ecto_sql 3.0.0) recently when doing deployments running ecto migrate or seed for some reason causes :econnrefused. I’m certain the db is running and the port is actually open at 5432:

my_app=# SELECT *
my_app-# FROM pg_settings
my_app-# WHERE name = 'port';
 name | setting | unit |                       category                       |                short_desc                | extra_desc |  context   | vartype | source  | min_val | max_val | enumvals | boot_val | reset_val | sourcefile | sourceline | pending_restart 
------+---------+------+------------------------------------------------------+------------------------------------------+------------+------------+---------+---------+---------+---------+----------+----------+-----------+------------+------------+-----------------
 port | 5432    |      | Connections and Authentication / Connection Settings | Sets the TCP port the server listens on. |            | postmaster | integer | default | 1       | 65535   |          | 5432     | 5432      |            |            | f


I also saw this in my docker log after a runtime crash. The db is definitely up and running. The log, config and release_tasks.ex can be found at: https://gist.github.com/mazz/9e649bc5f5b0339f5c03fcbf187bc669

This did not happen the last time I did a deployment and I’m not sure how to debug this. Any ideas?

> docker-compose run --rm my_app seed

Starting search-no-trigger_postgres_1 ... done
Starting dependencies..
Starting repos..
Running migrations for db repo: Elixir.DB.Repo migrations_path: /app/lib/db-0.1.0/priv/repo/migrations
21:53:17.251 [error] Postgrex.Protocol (#PID<0.148.0>) failed to connect: ** (DBConnection.ConnectionError) tcp connect (postgres:5432): connection refused - :econnrefused
21:53:17.251 [error] Postgrex.Protocol (#PID<0.147.0>) failed to connect: ** (DBConnection.ConnectionError) tcp connect (postgres:5432): connection refused - :econnrefused
21:53:19.357 [error] Postgrex.Protocol (#PID<0.148.0>) failed to connect: ** (DBConnection.ConnectionError) tcp connect (postgres:5432): connection refused - :econnrefused
21:53:20.094 [error] Postgrex.Protocol (#PID<0.147.0>) failed to connect: ** (DBConnection.ConnectionError) tcp connect (postgres:5432): connection refused - :econnrefused
21:53:20.194 [error] Could not create schema migrations table. This error usually happens due to the following:

  * The database does not exist
  * The "schema_migrations" table, which Ecto uses for managing
    migrations, was defined by another library

To fix the first issue, run "mix ecto.create".

To address the second, you can run "mix ecto.drop" followed by
"mix ecto.create". Alternatively you may configure Ecto to use
another table for managing migrations:

    config :db, DB.Repo,
      migration_source: "some_other_table_for_schema_migrations"

The full error report is shown below.

▸  Evaluation failed with: connection not available and request was dropped from queue after 2930ms. You can configure how long requests wait in the queue using :queue_target and :queue_interval. See DBConnection.start_link/2 for more information
▸      (ecto_sql) lib/ecto/adapters/sql.ex:590: Ecto.Adapters.SQL.raise_sql_call_error/1
▸      (elixir) lib/enum.ex:1327: Enum."-map/2-lists^map/1-0-"/2
▸      (ecto_sql) lib/ecto/adapters/sql.ex:677: Ecto.Adapters.SQL.execute_ddl/4
▸      (ecto_sql) lib/ecto/migrator.ex:470: Ecto.Migrator.verbose_schema_migration/3
▸      (ecto_sql) lib/ecto/migrator.ex:254: Ecto.Migrator.run/4
▸      (elixir) lib/enum.ex:769: Enum."-each/2-lists^foreach/1-0-"/2
▸      (elixir) lib/enum.ex:769: Enum.each/2
▸      (db) lib/db/release_tasks.ex:23: DB.ReleaseTasks.seed/1

Are you using the correct hostname for the database? Is your database container in the compose file named postgres?

Perhaps show the compose file?

Yeah, that might be it. The ecto:// DATABASE_URL might be malformed. I haven’t had time to investigate fully but I appended the compose file to the gist

I got it to deploy successfully. Some backstory: the elixir app is being migrated from a pyramid/python app and on top of that I’m adding a lot of schema to the db. Just before I deploy I need to pg_dump the development db to a file and then import it to the docker postgres instance on the digital ocean vps.

I just developed a search api, which uses a materialized view. I did refresh materialized view table_name; BEFORE I did pg_dump and when I deployed I got that weird database connection behavior.

This time, I materialized the view AFTER imported the db into the docker postgres instance. This all worked with no issue.

I really do not clearly understand why this is the case, maybe someone else here has ideas.

Anyway, thanks for being a sounding board at the very least :slight_smile:

Michael

2 Likes