Gigalixir Deployment Issue - '53300 (too_many_connections)' followed by 'Authenticity is not established by certificate path validation'

Questions / Help
, ,
1

Trying to deploy an existing app on Gigalixir and have had a bit of difficulty, but can’t find anything about how to resolve this current issue. It has no front end assets, it’s just an Absinthe GraphQL API.

This error occors:

 [error] Postgrex.Protocol (#PID<0.2308.0>) failed to connect: ** (Postgrex.Error) FATAL 53300 (too_many_connections) remaining connection slots are reserved for non-replication superuser connections

Followed by this warning:

[warn] Description: 'Authenticity is not established by certificate path validation' 

web.1  | Reason: 'Option {verify, verify_peer} and cacertfile/cacerts is missing'

I provisioned the database through the CLI as a Standard tier and set the pool size down to 2 as suggested by the troubleshooting docs for hitting the too_many_connections error.

Using Phoenix 1.6 / Elixir 1.12.3 / Erlang 24.2

config.exs:

import Config

config :evora,
  ecto_repos: [Evora.Repo],
  generators: [binary_id: true]

# Configures the endpoint
config :evora, EvoraWeb.Endpoint,
  url: [host: "localhost"],
  render_errors: [view: EvoraWeb.ErrorView, accepts: ~w(json), layout: false],
  pubsub_server: Evora.PubSub

prod.exs:

import Config

config :evora, Evora.Endpoint,
  load_from_system_env: true,
  http: [port: {:system, "PORT"}],
  server: true,
  secret_key_base: "${SECRET_KEY_BASE}",
  url: [host: "${APP_NAME}.gigalixirapp.com", port: 443],

releases.exs:

import Config
config :evora, EvoraWeb.Endpoint,
  server: true,
  http: [port: {:system, "PORT"}], # Needed for Phoenix 1.2 and 1.4. Doesn't hurt for 1.3.
  url: [host: System.get_env("APP_NAME") <> ".gigalixirapp.com", port: 443]

runtime.exs:

import Config
if config_env() == :prod do
  config :evora, Evora.Repo,
    adapter: Ecto.Adapters.Postgres,
    database: "",
    ssl: true,
    url: System.get_env("DATABASE_URL"),
    pool_size: String.to_integer(System.get_env("POOL_SIZE") || "10")

  secret_key_base =
    System.get_env("SECRET_KEY_BASE") ||
      raise """
      environment variable SECRET_KEY_BASE is missing.
      You can generate one by calling: mix phx.gen.secret
      """

  config :evora, EvoraWeb.Endpoint,
    http: [
      ip: {0, 0, 0, 0, 0, 0, 0, 0},
      port: String.to_integer(System.get_env("PORT") || "4000")
    ],
    secret_key_base: secret_key_base
end

elixir_buildpack.config

elixir_version=1.12.3
erlang_version=24.2
2

Hi Sean,

Sometimes connections can take a little while to get cleaned up so my guess is lowering your pool size actually solved the issue, but took some time to clean up. Are you still seeing it even after waiting a little while?

For the certificate issue, ​I usually see this when the app is trying to make an HTTPS connection, but isn’t verifying the certificate. For example, if it’s configured with verify_none.
See No way to silence certificate path validation warning · Issue #5352 · erlang/otp · GitHub

Could that be what your app is doing? If not, perhaps you need to install certs. For example,

3

Hey Jesse,

Thank you for the reply. Those two errors appear resolved, but now it just displays:

 Readiness probe failed: dial tcp 10.56.8.162:4000: connect: connection refused
 Your app is failing health checks, which means it isn't listening on port 4000 yet.

I’ve restarted the server a couple times and it does not give any more verbose errors.