kodepett

kodepett

Ecto Query Timeout - will this configuration suffice?

I’ve been experiencing a bit of timeout in production with respect to my queries. I’ve going through the docs and a few resources here, I came up with the below configuration. Kindly advise if it will suffice for production.

Configuration file:

config :debitor, Debitor.Repo,
url: database_url,
pool_size: String.to_integer(System.get_env(“POOL_SIZE”) || “10”)
timeout: 60000

Repo call:

`Repo.all(query, [timeout: 120_000])`

Thanks.

Most Liked

dimitarvp

dimitarvp

db_connection is what ecto_sql steps on. It is the lower-level library that Ecto’s DB handling code (namely the ecto_sql library) is wrapping in a more convenient package for us.

And yes, you simply have to add queue_timeout and queue_target to the Repo configuration. If you expect long transactions then it’s worth to simply increase queue_target to something larger and ignore the other option. In my hobby and professional apps I usually bump it up all the way to 5000 (ms) to give the apps breathing room to wait for a connection from the pool in high-load conditions. Having Ecto error out in such a situation isn’t very useful to me so I first increase the queue_target and then start analysing why my transactions take so long.

Alternatively, you can just increase pool_size to a bigger number. I rarely accept the default of 10 and usually set it to 20.


EDIT: Here’s my Repo config in one of my hobbyist projects that does ingestion of public retail datasets:

config :retail, Retail.Repo,
  username: "postgres",
  password: "postgres",
  database: "retail_dev",
  hostname: "localhost",
  pool_size: 20,
  queue_target: 5000,
  migration_timestamps: [type: :utc_datetime_usec]
18
Post #5
benwilson512

benwilson512

Author of Craft GraphQL APIs in Elixir with Absinthe

I would probably leave the default timeout alone, and only increase it for specific Repo calls. Regularly needing to support queries up to 60 seconds seems like an issue.

kodepett

kodepett

It makes a lot of sense now. Thanks a lot for the awesome explanation as well as the configuration details. I will apply changes to my config file on production and monitor.

Last Post!

dimitarvp

dimitarvp

It’s that, plus me wondering if there’s a better way.

But I think not. The current solution gives you enough knobs to turn so as to effectively make the dropping of messages de facto impossible.

Where Next?

Popular in Questions Top

Qqwy
Original source of discussion: This topic on the Pragmatic Programmers’ Functional Web Development with Elixir, OTP, and Phoenix forum. ...
New
hariharasudhan94
Lets say I have map like this fetching from my database %{"_id" => #BSON.ObjectId<58eb1a7a9ad169198c3dXXXX>, "email" => ...
New
lanycrost
Hi everyone! I need implement if…else if…else condition from my elixir code, and anymore of this control flow structures not work proper...
New
Lily
In templates/appointment/index.html.eex: <%= for appointment <- @appointments do %> <tr> <td><%= appoi...
New
vrod
I am using the Starship cross-shell prompt – it seems pretty nice, but I get some errors: [WARN] - (starship::utils): Executing command ...
New
Patoshizzle
After calling mix ecto.create I get this error: 17:00:32.162 [error] GenServer #PID<0.412.0> terminating ** (Postgrex.Error) FATAL...
New
jason.o
In the code below, if the create action is not set to accept “extra_key” as an input, it errors out with a message shown above. Is there ...
New

Other popular topics Top

KronicDeth
Elixir plugin for JetBrain’s IntelliJ Platform (including Rubymine) This is a plugin that adds support for Elixir to JetBrains IntelliJ...
289 36820 110
New
Qqwy
Original source of discussion: This topic on the Pragmatic Programmers’ Functional Web Development with Elixir, OTP, and Phoenix forum. ...
New
hariharasudhan94
I would like to know what is the best IDE for elixir development?
New
aadeshere1
I have a another noob question about loop. Since elixir is immutable, while loop is not directly possible. total = 10 while total != 0 ...
New
msaraiva
Surface is an experimental library built on top of Phoenix LiveView and its new LiveComponent API that aims to provide a more declarative...
564 44265 214
New
siddhant3030
Hi, I have to write a raw query for one of my project. But till now I have used ecto queries and don’t have much experience writing raw ...
New

We're in Beta

About us Mission Statement