Losing connection with bad pool_size value

That really surprised me. I have simple demo page with just one table with 4 rows.
I decided to push new version to inspect some info directly in MyApp.ErrorView.
Here is what I got:

%{conn: %Plug.Conn{adapter: {Plug.Adapters.Cowboy.Conn, :...}, assigns: %{kind: :error, layout: false, reason: %DBConnection.ConnectionError{message: "connection not available because of disconnection"}, stack: [{DBConnection, :checkout, 2, [file: 'lib/db_connection.ex', line: 926]}, {DBConnection, :run, 3, [file: 'lib/db_connection.ex', line: 742]}, {DBConnection, :prepare_execute, 4, [file: 'lib/db_connection.ex', line: 584]}, {Ecto.Adapters.Postgres.Connection, :prepare_execute, 5, [file: 'lib/ecto/adapters/postgres/connection.ex', line: 78]}, {Ecto.Adapters.SQL, :sql_call, 6, [file: 'lib/ecto/adapters/sql.ex', line: 235]}, {Ecto.Adapters.SQL, :execute_and_cache, 7, [file: 'lib/ecto/adapters/sql.ex', line: 424]}, {Ecto.Repo.Queryable, :execute, 5, [file: 'lib/ecto/repo/queryable.ex', line: 127]}, {Ecto.Repo.Queryable, :all, 4, [file: 'lib/ecto/repo/queryable.ex', line: 40]}]}, before_send: [#Function<0.101282891/1 in Plug.CSRFProtection.call/2>, #Function<4.24272003/1 in Phoenix.Controller.fetch_flash/2>, #Function<0.131660147/1 in Plug.Session.before_send/2>, #Function<1.46238705/1 in Plug.Logger.call/2>], body_params: %{}, cookies: %{}, halted: false, host: "...", method: "GET", owner: #PID<0.1589.0>, params: %{}, path_info: [], path_params: %{}, peer: {{}, 40276}, port: 80, private: %{MyApp.Router => {[], %{}}, :phoenix_endpoint => MyApp.Endpoint, :phoenix_flash => %{}, :phoenix_format => "html", :phoenix_layout => false, :phoenix_pipelines => [:browser], :phoenix_route => #Function<1.48121830/1 in MyApp.Router.match_route/4>, :phoenix_router => MyApp.Router, :phoenix_template => "500.html", :phoenix_view => MyApp.ErrorView, :plug_session => %{}, :plug_session_fetch => :done}, query_params: %{}, query_string: "", remote_ip: {}, req_cookies: %{}, req_headers: [{"host", "..."}, {"user-agent", "Mozilla/5.0 (X11; Linux x86_64; rv:55.0) Gecko/20100101 Firefox/55.0"}, {"accept", "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8"}, {"accept-language", "pl"}, {"accept-encoding", "gzip, deflate"}, {"dnt", "1"}, {"upgrade-insecure-requests", "1"}, {"pragma", "no-cache"}, {"cache-control", "no-cache"}, {"x-forwarded-for", "..."}, {"connection", "close"}], request_path: "/", resp_body: nil, resp_cookies: %{}, resp_headers: [{"cache-control", "max-age=0, private, must-revalidate"}, {"x-request-id", "dtrmic2pvfkk76s7617u7eacor16m5ru"}, {"x-frame-options", "SAMEORIGIN"}, {"x-xss-protection", "1; mode=block"}, {"x-content-type-options", "nosniff"}], scheme: :http, script_name: [], secret_key_base: "zlor2fDie2UtQ8DSJkkxXQxjFu5h7nFgnSHTk2KbB4tmB1qbVdx2JXi4/JmbY542", state: :unset, status: 500}, kind: :error, reason: %DBConnection.ConnectionError{message: "connection not available because of disconnection"}, stack: [{DBConnection, :checkout, 2, [file: 'lib/db_connection.ex', line: 926]}, {DBConnection, :run, 3, [file: 'lib/db_connection.ex', line: 742]}, {DBConnection, :prepare_execute, 4, [file: 'lib/db_connection.ex', line: 584]}, {Ecto.Adapters.Postgres.Connection, :prepare_execute, 5, [file: 'lib/ecto/adapters/postgres/connection.ex', line: 78]}, {Ecto.Adapters.SQL, :sql_call, 6, [file: 'lib/ecto/adapters/sql.ex', line: 235]}, {Ecto.Adapters.SQL, :execute_and_cache, 7, [file: 'lib/ecto/adapters/sql.ex', line: 424]}, {Ecto.Repo.Queryable, :execute, 5, [file: 'lib/ecto/repo/queryable.ex', line: 127]}, {Ecto.Repo.Queryable, :all, 4, [file: 'lib/ecto/repo/queryable.ex', line: 40]}], view_module: MyApp.ErrorView, view_template: "500.html"}

As you can see I got:

%DBConnection.ConnectionError{message: "connection not available because of disconnection"}

The part that surprised me:

  1. locally works (as always :smile:)
  2. on server works when I stop it and run like: ./my_app console - it’s also in production environment, right?
  3. on normal edeliver run it randomly lose connection to database
  4. No error is logged to my_app/log/erlang.log.1, but ALL requests that successfully finished are logged.

I think that problem could be in my config/prod.exs:

config :makestry, MyApp.Repo,
  adapter: Ecto.Adapters.Postgres,
  database: "my_app_prod",
  password: "postgres",
  pool_size: 20,
  username: "postgres"

but I don’t know what should I change and/or add to this configuration.
At: Ecto.Adapters.Postgres I see some config variables for timeouts, but don’t know if should I set them or maybe it’s something different (every deploy takes lots of time for me especially upload part), so testing one change after second is not so fast way in my case.
Can you explain why I have connection to database problems only when I run my app by edeliver?

Update: I tried to decrease pool_size to 10 and 2, but this not work. Page is loading one 1 on 3 times.

3 Likes

ok, in some servers like Heroku or Amazon we need to change pool_size to 10
other errors was from my bad (all nodes tried to connect to it’s own database)

2 Likes

Wow, their pool size is only 10?! How weird… We run 100 at work…

1 Like

@OvermindDL1: I used free trier offer :smile:

1 Like

Hmm, well the hobby-dev (free) tier for postgres allows maximum 20 connections as stated in their article on it. The Phoenix docs on Heroku deployment also has this info, and recommends setting pool_size to 18 and use the spare 2 for administration tasks (migration, iex, etc.).

2 Likes

@bobbypriambodo: I just read on one github issue that someone set it to 10.

1 Like