Elixir rest api returning 504 Gateway Time-out

otp
phoenix
rest_api

#21

We dont have a reverse proxy, requests hit cloud flare which is just doing dns resolution, which resolves to aws load balancer where the vm is defined as the target. I only see a deregistration delay of 300 seconds defined for alb.

The database is an aws RDS postgresql and for it the cpu utilization for the instance hits 50% at peak and the write iops hits 6000. When I reindexed the tables the write iops came down. will try to monitor the server performance today to verify if I need to do autovaccum and reindex regularly


#22

The AWS LB is a reverse proxy. As well as cloudflare does some reverse proxying, it weren’t able to do it’s caching job for static assets otherwise.


#23

I’ve had it in the past where an AWS load balancer has not been working correctly and would return 504s depending on which node it hits. I’ve also had load balancers failing to serve HTTP/2 depending on the node hit. The point is sometimes these things break and it’s not typically somewhere you’d look. I’d recommend creating a new load balancer and seeing if you still have the issue.


#24

I noticed that when I do sql inserts around 20 rows all in one go using db client on a highly used table I start seeing the timeouts on the client, I do VACUUM FULL on the table but that does not seem to stop it


#25

what is idle timeout on the ELB?https://docs.aws.amazon.com/elasticloadbalancing/latest/application/application-load-balancers.html#connection-idle-timeout

And if this goes through cloudflare as well what is the idle timeout there?


#26

@outlog idle timeout on aws alb is 60 seconds, in cloudflare only cname is defined for aws alb