speeddragon

speeddragon

I’m doing some optimization into an application that uses Riak Core, GenStage, and Phoenix PubSub. I’ve set up a load test that takes the application near the pod restart limit (using Kubernetes) and notice a large quantity of busy_dist_port.

After some search, I found the +zdbbl flag in Erlang page with the default value of 1024. Notice that on Riak KV performance page, their default for Distribution Buffer is 32MB, so I’ve updated mine to 32 MB, and 99.9% of the monitor messages related to the busy_dist_port are gone.

The monitor logs that remains are the following:

[info] monitor busy_dist_port <0.7799.0> [{initial_call,{'Elixir.GenStage',init,1}},{almost_current_function,{ets,match,2}},{message_queue_len,0}] {#Port<0.364>,unknown}

Where the almost_current_function can have different values:

- {almost_current_function,{ets,match,2}}
- {almost_current_function,{erts_internal,dsend_continue_trap,1}}
- {almost_current_function,{ets,lookup_element,3}}
- {almost_current_function,{erlang,bif_return_trap,2}}
- {almost_current_function,{pg2,'-group_members/1-lc$^0/1-0-',1}}

Which I think is related to too much load, and probably nothing I can do.

The other one is for

[info] monitor large_heap <0.4933.0> [{name,'Elixir.pubsub_name.Adapter'},{initial_call,{'Elixir.Phoenix.PubSub.PG2',init,1}},{almost_current_function,{erlang,bif_return_trap,2}},{message_queue_len,771}]

With also similar different values in almost_current_function.

- {gen_server,handle_msg,6}
- {'Elixir.Phoenix.PubSub',local_broadcast,4}
- {erlang,bif_return_trap,2}
- {ets,lookup_element,3}

Is there some recommendation in optimization that can be done that I’m missing? These should be the standard monitor messages when the server in under extreme load, right?

EDIT: The pool_size used for PubSub in this project was set to 10 for a 2.5 core machine (2500m) when the documentation of PubSub recommends 1 partition per 4 cores. Change to a proper number (1) the large_heap monitor logs produced disappeared.

Where Next? Top

Trending in Discussions Top

AstonJ
As the title says, please share what you’ve been up to with Elixir. Whether that’s been learning it, looking into it, making stuff with i...
2977 94592 917
New
cblavier
Hey there, It’s been more than a year since we started using LiveView as our main UI library and building a whole library of UI componen...
New
mudasobwa
I am happy to introduce the very α version of the new programming language compiled to BEAM. Welcome Cure. It has literally three kille...
New
heathen
Quite interesting article Google brought me. Didn’t find any mentions about it here. What do you think in general? Would you use togethe...
New
maennchen
:warning: Security advisory: Decimal DoS vulnerability A vulnerability has been published for decimal where very large exponents can cau...
New
marciol
It would be helpful to have a list of companies worldwide that hire engineers without prior experience in Elixir. Often, it can be quite ...
New
durvia
Anyone running long-lived stateful processes on BEAM? We’re building an AI agent runtime and would love to compare notes. We’re a small ...
New

Other Trending Topics Top

marciok
Hi there! We created Gust: A task orchestrator inspired by Airflow. For those who have never heard about Aiflow, it’s a Python-based wor...
New
jimsynz
Beam Bots (or just BB for short) is a framework for building fault-tolerant robotics applications in Elixir using familiar OTP patterns. ...
New
Dmk
Xamal is a deployment tool for Elixir apps that deploys native releases to bare metal servers over SSH. It’s a port of GitHub - basecamp/...
New
netoum
Corex is an accessible, unstyled UI component library for Phoenix that integrates Zag.js state machines using Vanilla JavaScript and Live...
New
webofbits
With AI doing more of the implementation work, I’ve been wondering how much coding I should deliberately keep doing myself. My main conc...
#ai
New
webofbits
Aludel - LLM Evaluation Workbench Aludel is an embeddable Phoenix LiveView dashboard for evaluating and comparing LLM prompts across mult...
New

We're in Beta

About us Mission Statement

Options

Thread Display Mode




Thread Preview

Skip Thread Previews