Hello,

I’m having some trouble optimizing the memory usage for my application. Its job is manage and collect data from a bunch of networked devices (typical IOT application).

I create a process for each managed device (via DynamicSupervisor) that handles communication and data retrieval. I send the data (a map) to an Agent process that accumulates the data in a giant map of device address => device data. Once every few seconds, another process grabs this accumulated data (if present), encodes it with JSON.encode!/1 and publishes it on MQTT. When the data retrieval happens, the Agent resets its internal state to an empty map via get_and_update.

This works fine, but for whatever reason the agent accumulates binary memory over time. :recon.proc_count(:memory, 10) shows its memory constantly increasing, even though the data it collects should be temporary. I can call :recon.bin_leak/1 to clear the memory out, but I’d rather find out what’s causing the constant memory increases. For reference, the number of devices is less than 1000 and the data accumulated for each device can be enough to push the outgoing encoded message to ~1MB.

What might I be doing incorrectly here? I’d rather not run garbage collection manually, but I can do so if necessary. Thanks!