We are using Horde to have “about one” process for a key in a cluster. We poll external service and each run we ask Horde to start a process with a key. If we already saw the key, Horde reports already_started as expected. However, we observe that internal merkle_map grows linearly with startup attempts.
crdt_size = fn -> pid = Process.whereis(Elixir.MyHordeSupervisor.Crdt) state = :sys.get_state(pid) map_size(Map.get(state, :merkle_map).map) end iex(9)> crdt_size.() 10077 iex(11)> crdt_size.() 10077 iex(12)> Enum.each(1..10000, fn _ -> start_constant_queue.() end) :ok iex(13)> crdt_size.() 20079 iex(15)> Enum.each(1..10000, fn _ -> start_constant_queue.() end) :ok iex(16)> crdt_size.() 30081
depending on number of keys and refresh rate memory limit hit pretty quickly.
Elixir.MyHordeSupervisor.Crdt process memory size will be hundreds of megabytes.
While writing code we following the docs on hexdocs.
Any help would be appreciated. Thanks!