blackode
Issue: Response count mismatch in decode function under concurrent load
Setup: 4-node cluster, 500 concurrent HTTP requests dispatched via Task.async. All requests return within 2ms max latency.
Problem: Instrumented the decode function with entry/exit log markers. Entry log shows all 500 invocations as expected, but exit log shows only ~179 completions — roughly 64% of decode calls are not reaching the exit log point.
I am using Task.yield_many(spec.kill_cutoff_time_ms) with 4.5 seconds as timeout here. Each task will make http request and decode call Jason.decode
I have tried with Nimble ParserPool but I could reach only 232 at maximum on 64 Core system with pool size of
200
Here is the ParserPool.
defmodule GrncHub.ParserPool do
@behaviour NimblePool
require Logger
@pool_size 500
def child_spec(opts) do
%{
id: **MODULE**,
start: {**MODULE**, :start_link, \[opts\]},
type: :worker,
restart: :permanent,
shutdown: 500
}
end
def start_link(\_) do
NimblePool.start_link(worker: {**MODULE**, nil}, pool_size: @pool_size, name: **MODULE**)
end
@impl true
def init_worker(pool_state), do: {:ok, nil, pool_state}
@impl true
def handle_checkout(:parse, \_from, worker_state, pool_state) do
{:ok, worker_state, worker_state, pool_state}
end
@impl true
def terminate_worker(\_reason, _worker_state, pool_state) do
{:ok, pool_state}
end
@doc """
Returns {:ok, decoded} | {:error, :invalid_json} | {:error, :worker_crashed} | {:error, :invalid_input}
"""
def parse(payload, content_type, parser) when is_binary(payload) do
parser = default_parser(content_type, parser)
try do
NimblePool.checkout!(
__MODULE__,
:parse,
fn _from, worker_state ->
{GrncHub.Parser.decode(payload, content_type, parser), worker_state}
end,
:infinity
)
catch
:exit, reason ->
Logger.error("ParsePool worker crashed: #{inspect(reason)}")
{:error, %{"error" => "worker_crashed"}}
end
end
def parse(\_invalid), do: {:error, %{"error" => "worker_crashed"}}
defp default_parser(:json, nil), do: Jason
defp default_parser(\_content_type, parser), do: parser
end
Where Next?
Trending in Discussions
Other Trending Topics
Latest on Elixir Forum
Sponsor Spotlight
Phoenix, LiveView, multimedia pipelines, distributed systems. 400 engineers, Elixir since 2017. Available for hire.
Categories:
Sub Categories:
Forums
Popular Tags
- #ecto
- #liveview
- #troubleshooting
- #learning-elixir
- #library
- #deployment
- #erlang
- #testing
- #genserver
- #mix
- #absinthe
- #remote-other
- #otp
- #plug
- #how-to-question
- #macros
- #postgres
- #elixirconf
- #channels
- #exunit
- #discussion
- #code-sync
- #podcasts
- #javascript
- #onsite
- #dialyzer
- #docker
- #authentication
- #umbrella
- #full-time-contract
- #podcasts-by-brainlid
- #ai
- #ecto-query
- #elixirconf-us
- #blog-post
- #elixir-ls
- #phoenix_html
- #iex
- #graphql
- #genstage
- #websockets
- #supervisor
- #advent-of-code
- #distillery
- #processes
- #elixirconf-eu
- #api
- #forms
- #metaprogramming
- #hex
Our Sponsors
Build Elixir applications with speed and confidence.
Supporting innovation across the BEAM ecosystem.
We build reliable cloud platforms for business-critical systems.
Catch errors, track performance, monitor hosts and more.
Error tracking, logs, uptime, and Just Enough APM. Built for developers.
Master modern tools, write cleaner code, and tackle real-world challenges.
Producing high quality Elixir screencasts since 2017.
The deployment platform built for Elixir: PaaS ease, VPS control.
The team behind Membrane, Popcorn, LiveDebugger. Available for hire.
Courses that'll move you from confusion to "Aha, now I get it!"
Showing Posts 1 to 10- Show Best Posts
- Show All (oldest first)
- Show All (newest first)
Schultzer
Are you doing actual HTTP request or just benchmarking 500 concurrent processing decoding JSON, I could imagine that the bottleneck being the pool or another place in the framework. There is quite a few places that you can create scheduler starvation and mailbox contention if you don’t use the receive optimization.
But without complete reproducible example no one can help you.
andyleclair
Since JSON parsing is a CPU-bound task, it’s better to parse the JSON in whatever task process is making the HTTP request, rather than in another set of tasks, since all the data would have to be copied from the HTTP task to the JSON task. Does that make sense? Unfortunately you can’t just throw tasks at things and expect it will magically get faster, and processes have a cost. They’re cheap but not free.
blackode
Thanks for the prompt response.
I have seen that my cpu can reach maximum of 42 % with 200 pool connections for decoding.
I am doing parsing inside the main task which is making http request and as soon as response received, I am calling
ParserPool.parse()My payloads are around > 1MB. I am unable to figure this out. Is there any issue with approach? Glad if you can suggest alternative.blackode
Yes, I’m doing actual HTTP requests. I don’t have an example one to share. It is happening in live and I’m looking for suggestions like how can we achieve the decoding faster. My payloads are around > 1MB.
I am unable to figure this out. Is there any issue with my approach? Glad if you can suggest alternative.
Thanks
beepbeepbopbop
From
nimblepoolREADME in their Github:I assume you have a single worker pool as you’ve provided an atom name for this process.
blackode
@beepbeepbopbop thanks for prompt reply. I will create one per scheduler and will update you.
andyleclair
You simply do not need a pool of parser processes. You don’t gain anything by parallelizing the parsing, simply call
JSON.decode!. A pool of processes can help you if the thing you’re doing is IO bound, however, as I stated, JSON parsing is CPU bound and the work cannot be parallelized in that way. You will actually make performance worse by doing this.Asd
How are you making these HTTP requests? Do you have an example of your request, decode task loop?
dimitarvp
With a C/C++/Rust NIF anything below 50MB should be in the ballpark of milliseconds. Don’t optimise for a problem you don’t have. Just parse them in place and don’t worry about it.
Also you might be hitting a default timeout somewhere – in your original post.
saleyn
Take a look at glazer, which is several times more efficient at parsing large JSON payloads compared to Jason.