Connection refused for Postgrex.Notifications.start_link/1 on gitlab ci

ecto

#1

Postgrex.Notifications.start_link/1 connects and works well on localhost

Relevant parts of my .gitlab-ci.eml:

image: bitwalker/alpine-elixir:latest

stages:
  - setup
  - test

prepare:
  stage: setup
  cache:
    key: elixir-deps
    paths:
      - "_build"
      - "deps"
  script:
    - elixir -v
    - mix local.rebar --force
    - mix local.hex --force
    - mix deps.get

test:
  stage: test
  services:
    - mdillon/postgis:10-alpine
  variables:
    MIX_ENV: test
    DATABASE_URL: postgres://postgres:@mdillon__postgis/db_test
  script:
    - apk add postgresql-client
    - mix deps.get --only test
    - mix compile
    - mix ecto.setup # runs create/load/migrate successfully
    - mix test
  cache:
    key: elixir-deps
    paths:
      - "_build"
      - "deps"
    policy: pull

Note that mix ecto.setup runs successfully, and then

...
Generated web app

$ mix ecto.setup
config DATABASE_URL: "postgres://postgres:@mdillon__postgis/db_test"
The database for RDB.Repo has been created
The structure for RDB.Repo has been loaded from /builds/.../apps/rdb/priv/repo/structure.sql

$ mix test
config DATABASE_URL: "postgres://postgres:@mdillon__postgis/db_test"
==> rdb
pg_config: [
  adapter: Ecto.Adapters.Postgres,
  types: RDB.PostgrexTypes,
  pool: Ecto.Adapters.SQL.Sandbox,
  pool_size: 20,
  url: "postgres://postgres:@mdillon__postgis/db_test"
]
PGListener.start_link: {:ok, #PID<0.343.0>}
13:08:39.993 [error] GenServer #PID<0.343.0> terminating
** (stop) %DBConnection.ConnectionError{message: "tcp connect (localhost:5432): connection refused - :econnrefused"}
Last message: nil
** (Mix) Could not start application web: Web.Application.start(:normal, []) returned an error: shutdown: failed to start child: Web.PGListener
    ** (EXIT) %DBConnection.ConnectionError{message: "tcp connect (localhost:5432): connection refused - :econnrefused"}
ERROR: Job failed: exit code 1

The source code is almost identical to https://github.com/idi-ot/notify_and_channels

For some reason PGListener.start_link: {:ok, #PID<0.343.0>} pg_listener starts during database tests, and not web tests … But it’s the same on localhost where tests pass.


#2

If I separate tests for each app, the tests for rdb pass, while web tests still fail

$ cd apps/rdb && mix test && cd ../..
config DATABASE_URL: "postgres://postgres:@mdillon__postgis/db_test"
.....

Finished in 0.5 seconds
5 tests, 0 failures

Randomized with seed 64824
xUnit - 5 tests, 0 failures in 0.5 seconds a

$ cd apps/web && mix test && cd ../..
config DATABASE_URL: "postgres://postgres:@mdillon__postgis/db_test"
pg_config: [
  adapter: Ecto.Adapters.Postgres,
  types: RDB.PostgrexTypes,
  pool: Ecto.Adapters.SQL.Sandbox,
  pool_size: 20,
  url: "postgres://postgres:@mdillon__postgis/db_test"
]
PGListener.start_link: {:ok, #PID<0.379.0>}
13:27:49.343 [error] GenServer #PID<0.379.0> terminating
** (stop) %DBConnection.ConnectionError{message: "tcp connect (localhost:5432): connection refused - :econnrefused"}
Last message: nil
** (Mix) Could not start application web: Web.Application.start(:normal, []) returned an error: shutdown: failed to start child: Web.PGListener
    ** (EXIT) %DBConnection.ConnectionError{message: "tcp connect (localhost:5432): connection refused - :econnrefused"}
ERROR: Job failed: exit code 1

#3

Does your web app have its own config/test.exs that overrides the config with localhost?


#4

No, here’s the web’s test config

use Mix.Config

# We don't run a server during test. If one is required,
# you can enable the server option below.
config :web, Web.Endpoint,
  http: [port: 4001],
  server: false

# Print only warnings and errors during test
config :logger, level: :warn

and neither do I do anything for test env in endpoint.ex

  @doc """
  Callback invoked for dynamically configuring the endpoint.

  It receives the endpoint configuration and checks if
  configuration should be loaded from the system environment.
  """
  @spec init(atom, Keyword.t()) :: {:ok, Keyword.t()} | no_return
  def init(_key, config) do
    if config[:load_from_system_env] do
      port = System.get_env("PORT") || raise("expected the PORT environment variable to be set")

      secret_key_base =
        System.get_env("SECRET_KEY_BASE") ||
          raise("expected the SECRET_KEY_BASE environment variable to be set")

      # inet6?
      config =
        config
        |> Keyword.put(:http, [:inet6, port: port])
        |> Keyword.put(:secret_key_base, secret_key_base)

      {:ok, config}
    else
      {:ok, config}
    end
  end

#5

Oh, I’ve missed localhost in localhost:5432! Now I have at least some idea of where to look …

Just in case, here’s rdb's app config/test.exs

use Mix.Config

config :logger, level: :warn

config :rdb, RDB.Repo,
  pool: Ecto.Adapters.SQL.Sandbox,
  pool_size: 20

if url = System.get_env("DATABASE_URL") do
  IO.inspect(url, label: "config DATABASE_URL")
  config :rdb, RDB.Repo, url: url
else
  config :rdb, RDB.Repo,
    database: "db_test",
    hostname: "localhost"
end

and repo.ex

  @doc """
  Callback invoked for dynamically configuring the repo.

  It receives the repo configuration and checks if
  configuration should be loaded from the system environment.
  """
  @spec init(atom, Keyword.t()) :: {:ok, Keyword.t()} | no_return
  def init(_type, config) do
    if config[:load_from_system_env] do
      db_url =
        System.get_env("DATABASE_URL") ||
          raise("expected the DATABASE_URL environment variable to be set")

      config = Keyword.put(config, :url, db_url)

      {:ok, config}
    else
      {:ok, config}
    end
  end

#6

Your web app doesn’t seem to have any configuration for the Repo. Can that be a problem? Or should it pick them up from rdb?


#7

web depends on rdb ({:rdb, in_umbrella: true}), so, yes, I expected it to pick it up from rdb. It does that when I run tests on my laptop, at least.


#8

It seems like postgrex doesn’t recognize :url (which is just for ecto?) option and falls back to its defaults. There is probably a function in ecto which parses the url and takes postgrex options out of it. Will try to use that.


#9

Using

pg_config = RDB.Repo.config()
Postgrex.Notifications.start_link(pg_config)

instead of

pg_config = Application.get_env(:rdb, RDB.Repo)
Postgrex.Notifications.start_link(pg_config)

when

config :rdb, RDB.Repo,
  pool: Ecto.Adapters.SQL.Sandbox,
  pool_size: 20,
  url: System.get_env("DATABASE_URL")

worked.