I am developing an API that fetches data from multiple remote databases. For each of these databases, an Ecto.Repo is configured at runtime when the application starts.
When I am trying to fetch data from a disconnected database (network failure, database server not started, …), I get this error after a timeout:
** (DBConnection.ConnectionError) connection not available and request was dropped from queue after 2657ms. This means requests are coming in and your connection pool cannot serve them fast enough. You can address this by:
1. Tracking down slow queries and making sure they are running fast enough
2. Increasing the pool_size (albeit it increases resource consumption)
3. Allowing requests to wait longer by increasing :queue_target and :queue_interval
See DBConnection.start_link/2 for more information
Since a pool of connection is managed in the background, when I am attempting to query a disconnected database, would it be possible to get an immediate error indicating that no connection is available rather than having to wait around 2657ms?
Also, I would like my API to offer the possibility to get the status (online or disconnected) of each database without having to wait for a 2657ms timeout. Any idea if Ecto.Repo allows that?
You might be misreading the error message. It says you are sending too much connection requests to the pool of the particular Repo and the pool is thus timing out.
Can you check how many connections you configure per Repo?
The message seems to indicate otherwise but it is exactly that error that I get while trying to query a repo that don’t have any connection.
I left the default so there are supposed to be 10 connections per pool/repo but there are not any connection when the database is unreachable.
For the purpose of my tests, I configure a repo that points to an non-existing (MS SQL) database server and I see the following messages repeating indefinitely in the console:
My config for my simulated unreachable database server was using localhost as host. I changed the config to target a real database server which I turned off and here are the errors (which also repeat indefinitely in the console):
Yes the “unreachable” database server is in the local network, behind a firewall. But it is unreachable on purpose. My goal is to write a test that simulate an unreachable database server.
My test already works but it takes around 2657ms to return an answer. I would like a way to return an immediate response indicating that the database is not available when there are no connection in the pool, rather than trying anyway and waiting for it to timeout.
I’d think Ecto.Repo is not designed with this in mind but what is stopping you from giving it a much smaller timeout, say 50-100ms? That should be enough to have tests failing rather fast.
I seem to have misread your requirement for a “deliberately failing but you want it to fail fast” test. Apologies.
Changing :queue_interval in DBConnection to 100ms (instead of the default 1000ms) seems to do the trick. Thank you!
However, I must admit that I do not fully understand the explanation about :queue_target and :queue_interval in the documentation.
I have 10 (failing) connections in my pool, queue_target: 50 and queue_interval: 100. I cannot come up with a theoretical time it would take for a request to be dropped from the queue.
Tests show results varying between 111ms and 202ms which is more acceptable than 2657ms in my use case.
