mpugach
There is a piece of Elixir software that can generate some network load GitHub - mpugach/wdbomber-eli at 0.1.2 · GitHub
It utilise Flow to dispatch many concurrent requests
We use it to load test or Phoenix API
The specifics is that 10-300 seconds! response is normal, but we need to handle a lot of them at the same time
For some reason about 300 of 500 requests almost immediately return with {:error, %HTTPoison.Error{id: nil, reason: :closed}} with none log entries on Phoenix side in development environment
The Ruby production server (which we want to replace) fails only for 10 responses
here is the endpoint config
http: [
port: System.get_env("PORT") || 4000,
protocol_options: [idle_timeout: 1_300_000, inactivity_timeout: 1_300_000, max_keepalive: 5_000_000],
timeout: 1_300_000,
transport_options: [num_acceptors: 10_000, max_connections: :infinity]
]
so, what do we miss?
Where Next?
Trending in Questions
Other Trending Topics
Latest Phoenix Threads
Latest on Elixir Forum
Sponsor Spotlight
Catch errors, track performance, monitor hosts and more.
Categories:
Sub Categories:
Forums
Popular Tags
- #ecto
- #liveview
- #troubleshooting
- #learning-elixir
- #deployment
- #library
- #erlang
- #testing
- #genserver
- #mix
- #absinthe
- #remote-other
- #otp
- #plug
- #how-to-question
- #macros
- #postgres
- #channels
- #elixirconf
- #exunit
- #discussion
- #code-sync
- #javascript
- #podcasts
- #onsite
- #dialyzer
- #docker
- #authentication
- #umbrella
- #full-time-contract
- #podcasts-by-brainlid
- #ecto-query
- #elixir-ls
- #blog-post
- #phoenix_html
- #iex
- #graphql
- #ai
- #genstage
- #elixirconf-us
- #websockets
- #supervisor
- #advent-of-code
- #distillery
- #processes
- #api
- #forms
- #metaprogramming
- #security
- #hex
Our Sponsors
Build Elixir applications with speed and confidence.
Supporting innovation across the BEAM ecosystem.
We build reliable cloud platforms for business-critical systems.
Catch errors, track performance, monitor hosts and more.
Error tracking, logs, uptime, and Just Enough APM. Built for developers.
Practical resources that improve the lives of professional developers.
Bringing learning, work and compliance together since 1987.
Producing high quality Elixir screencasts since 2017.
The team behind Membrane, Popcorn, LiveDebugger. Available for hire.
Courses that'll move you from confusion to "Aha, now I get it!"
Showing Posts 1 to 5- Show Best Posts
- Show All Posts (oldest first)
- Show All Posts (newest first)
kodepett
HTTPoison uses connection pool, I think the above error is due to your pool size, once all connections with the pool are use, the above error is inevitable.
mpugach
Thank you @kodepett for the hint
I specify the pool size here wdbomber-eli/lib/wdbomber/client.ex at 0.1.5 · mpugach/wdbomber-eli · GitHub
so it should be the same as the number of concurrent workers
still, some amount of requests fail with the same error
kodepett
How about disabling connection pooling - open a connection on the fly; I understand the performance gain when using a pool(tcp handshake etc) but to help us isolate the problem. Let’s see if the error will be cleared then we can focus on the pool or a different area. My two cents sir.
mpugach
Thank you, it fixed the issue
kodepett
We have to look into the connection pool issue and see what can do differently to avoid the error, I’ve seen a lot discussions on the connection pool. If you are able to look into it, let me know. Thanks a lot.