Now that I increased the pool it helped raise the bar so I have to wait again until I see the new level at which it breaks :S
When I find the high water mark I will try that
9mm
9mm
benwilson512
Author of Craft GraphQL APIs in Elixir with Absinthe
Ahhhh hah! I figured there was another pool in there. How are you configuring it now? For the number of CPUs you’ll have it may be important to use a very large pool. How many domains are you hitting btw?
1
9mm
Each API has its own domain, so 2-3… although it would be amazing to set more. I notice if I enable more than 2 API’s it dies exponentially more quickly so I have just had it set to 2 for now. If I could set 5 or more that would be insanely awesome, but I’ll take what I can get.
Here’s how I set it in application.ex
:hackney_pool.start_pool(:api, [timeout: 1000, max_connections: 200])
I would like to take to totally ask what the "napkin math’ is for setting up pools…
like how do I find the best values (ideally without testing constnatly which is a bit of a pain. Perhaps what math do you use to calculate good ‘starter’ values), taking into consideration things like
- total outgoing requests / second
- total hosts
- timeout value set for each host
- pool size (hackney
max_connections) - “time we keep connections alive in the pool” (hackney
timeout)
0
benwilson512
Author of Craft GraphQL APIs in Elixir with Absinthe
I’m not sure I understand, what does it mean to “enable” an API?
My back of the napkin math would be to leave the timeout alone, and then set the connections to like 800. Reason is that if the 50 is a default for ~2-4 cores then at 64 cores you want at least 800.
Practically speaking though the ideal value is going to need to be found empirically. I’d test 400, 800, 1200, 1600 and see how it goes.
1
9mm
Oh, the outgoing requests are data apis. for each incoming request im basically aggregating data from all simultaneous requests.
So 1 outgoing request = 1 api = 1 hostname. I can enable/disable them, currently its set to 2
0
9mm
I can’t believe I got duped by that stupid default pool setting in Tesla. Without a pool I was thinking I was magically getting ‘more connections’ by multiplying the app but really I was just multiplying max_connections
I wonder what the upper limit are for max_connections. Like if I put it at 5000 what actually would limit that (like that the OS level etc)? CPU? ulimit? what else? ip_local_port_range?
1
benwilson512
Author of Craft GraphQL APIs in Elixir with Absinthe
Really your best bet is to just benchmark. Test at some of the levels I suggested and see if you see a pattern, then do larger jumps to see how far that pattern goes.
0
9mm
Ok thanks, I’ll do that tomorrow.
Fast question regarding something you said above. You said to give kubedns an extra cpu or two. Do I do that just by subtracting 2 CPU’s from whatever my server is? so if its 64 then I set limits and requests to 62000m, or should those be 64000m and then I set kubedns elsewhere?
Also, I do remember testing pool settings for almost 2 full days (and other HTTP libraries). I remember on buoy when I went past 300-500 connections in the pool they wouldnt even connect… but perhaps because on gigalixir everything is 8cpu’s max, so it wouldnt be able to run on one machine ever.
This was the first time testing Pool settings on 64 cpu server. That would have been one of the first things i test in the new environment except i got sidetracked for a day or 2 because of the hidden pool.
Either way, I’ll try to see how high I can go now on one machine for the number in the pool before stuff starts to not even connect.
1
9mm
Second follow up question. You said I would have to ‘trace a function’. What’s the best tool to actually do that? I see like 10 versions of different ‘prof’ like cprof, dprof, eprof, etc etc. I have no idea which one would be best for this use case (I would probably trace the very top of the execution function ‘tree’, with the Task.async and Task.await, and trace all 3 functon calls inside the task). Id imagine it would trace the execution time of every inner/nested function, and maybe extra info too?
0