Unfortunately our solution here is not LiveView based but HTTP API based =/
victorolinasc
chasers
Ah … yeah I had an API product for years that was designed to be async. So people would POST jobs with a callback URL. We’d hit that with results, or with an endpoint to gather results when the job was finished. It worked well and people understood why we had to do that based on the type of work we were doing.
Other than that, I would limit concurrency with a pool. So the whole system can have N number of concurrent jobs. You don’t even need poolboy for this … just a Dynamic Supervisor limiting the number of children.
I personally wouldn’t try and base it off of CPU. I’m sure it can be done I guess but I’d start with concurrency limiting I think to play with that. Maybe then have a failsafe based on CPU if you want to push the boundaries a bit more.
2
dom
In that case I think you could just use a big enough pool (say 2 * core count) for low priority requests, and do the work directly in the request handler for high priority requests, bypassing the pool.
Reject low priority requests if the pool is full. You can also use s_broker if you want to get fancy and allow some queuing to deal with sudden spikes.
Even if the pool is saturated and CPU usage reaches 100%, the app will stay responsive, since the BEAM context switches very frequently.
0
victorolinasc
Thanks! That is my current approach and it works quite well with the only caveat of having to “guess” the proper pool size for each environment.
The “normal” schedulers context switch very frequently but dirty_cpu schedulers might not do that (depending on the NIF implementation) so I am trying to be safe here and not ever allowing it to reach 100%.
0
drakkhenn
I think I would have first used a “simple challenge” just to avoid “script kiddies” then your “annoying algo”. If you have a lot of users, to a specific cpu → server → hardware (GPU).
0
victorolinasc
Thanks! That is also a nice idea. Our “challenge” was an external service that went down (Google 
).
We are testing other on-premises fallbacks . There are some interesting alternatives currently…
Thanks for your suggestion!
0
cmkarlsson
I have used jobs (GitHub - uwiger/jobs: Job scheduler for load regulation · GitHub) for these kind of things before. It allows you to have a queue which is regulated based on CPU sampling for example.
Or you can use a “circuit breaker” library to deal with this. They generally implement the correct algorithms for this. I’ve used fuse (GitHub - jlouis/fuse: A Circuit Breaker for Erlang · GitHub) in the past but it hasn’t had any updates the last couple of years so don’t know if it will work with the latest erlang versions.
0
victorolinasc
Awesome! Thanks a lot for the info! Will try to play with uwieger/jobs A very interesting library to have under my set of tools.
Thank you all for your replies. We have tested the process pool approach and it is handling it pretty good up to now. Though the subject keeps interesting me and I will try other approaches.
1