Creeping :ets tables

Questions / Help
1

While monitoring memory usage, an :ets.i revealed what is possibly a leak in my code. I have a GenServer named WexStore.RemoteJobScheduler which dynamically creats :dets files then closes them, typically in that fashion:

		try do
			:dets.open_file( :remote_write, [
				{:file, to_charlist( job_path ) },
				{:type, :set},
				{:access, :read_write} ] )
			:dets.insert :remote_write, { :from_node, from_node }
		after
			:dets.close :remote_write
		end

And typically nothing shows up on :ets.i
But on production, after a while I see :ets entries where id is a ref, name is always “rules” and the owner is my GenServer. They don’t take much memory but I’m conscious of a limited number of :ets tables in the VM and after all I am not explicitly creating them.

iex(wex@wex_s1)1> :ets.i
 id              name              type  size   mem      owner
 ----------------------------------------------------------------------------
(....)
 #Ref<0.3473999207.3311009794.192225> rules             set   0      305      'Elixir.WexStore.RemoteJobScheduler'
 #Ref<0.3473999207.3311534082.229174> rules             set   0      305      'Elixir.WexStore.RemoteJobScheduler'
 #Ref<0.3473999207.3311796225.222327> rules             set   0      305      'Elixir.WexStore.RemoteJobScheduler'
 #Ref<0.3473999207.3311796226.126516> rules             set   0      305      'Elixir.WexStore.RemoteJobScheduler'
 #Ref<0.3473999207.3312320514.142647> rules             set   0      305      'Elixir.WexStore.RemoteJobScheduler'
 #Ref<0.3473999207.3313106945.20359> rules             set   0      305      'Elixir.WexStore.RemoteJobScheduler'
 #Ref<0.3473999207.3313369090.106714> rules             set   0      305      'Elixir.WexStore.RemoteJobScheduler'
 #Ref<0.3473999207.3313631233.74447> rules             set   0      305      'Elixir.WexStore.RemoteJobScheduler'
 #Ref<0.3473999207.3313893377.227474> rules             set   0      305      'Elixir.WexStore.RemoteJobScheduler'
 #Ref<0.3473999207.3314941953.231154> rules             set   0      305      'Elixir.WexStore.RemoteJobScheduler'
 #Ref<0.3473999207.3315204097.185773> rules             set   0      305      'Elixir.WexStore.RemoteJobScheduler'
 #Ref<0.3473999207.3317039105.8236> rules             set   0      305      'Elixir.WexStore.RemoteJobScheduler'
 #Ref<0.3473999207.3317563394.94553> rules             set   0      305      'Elixir.WexStore.RemoteJobScheduler'
 #Ref<0.3473999207.3317825537.32266> rules             set   0      305      'Elixir.WexStore.RemoteJobScheduler'
 #Ref<0.3473999207.3317825538.130207> rules             set   0      305      'Elixir.WexStore.RemoteJobScheduler'
 #Ref<0.3473999207.3319660545.180154> rules             set   0      305      'Elixir.WexStore.RemoteJobScheduler'

I could not find information about this. Could any :ets expert shed some light on the meaning of these :ets entries?

3 Likes
2

Something in your code, or some library you call, is creating them.

You could use tracing to see which module is creating the ets table. In Erlang I would do:

> dbg:tracer(),dbg:p(whereis('Elixir.WexStore.RemoteJobScheduler'),[call]),dbg:tp(ets,new,cx).

The above starts a tracer, traces the process called ‘Elixir.WexStore.RemoteJobScheduler’ for calls, and then enables call (c) and exception (x) tracing on ets:new. Not sure what the equivalent in Elixir is.

10 Likes
3

Hah, that’s actually super useful. Had no clue I could do something like it, thanks!

2 Likes
4 
:dbg.tracer()
:dbg.p(Process.whereis(WexStore.RemoteJobScheduler),[:call])
:dbg.tp(:ets,:new,:cx).
6 Likes
5

Thank you very much.

The culprit is xmerl_scan. It is used by the AWS library ex_aws via sweet_xml.

(<0.5968.0>) call ets:new(rules,[set,public]) ({xmerl_scan,initial_state,2})
(<0.5968.0>) returned from ets:new/2 -> #Ref<0.158833462.935723010.69902>

That library crashes from time to time due to incorrectly formatted xml (I believe network-related hiccups) and must not clean up after itself.

[error] 2564- fatal: {:error, {:unexpected_end_of_STag}}
[error] GenServer WexStore.RemoteJobScheduler terminating
** (stop) {:fatal, {{:error, {:unexpected_end_of_STag}}, {:file, :file_name_unknown}, {:line, 2}, {:col, 16839}}}
(xmerl 1.3.26) xmerl_scan.erl:4124: :xmerl_scan.fatal/2
(xmerl 1.3.26) xmerl_scan.erl:2133: :xmerl_scan.scan_element/12
(xmerl 1.3.26) xmerl_scan.erl:2605: :xmerl_scan.scan_content/11
(xmerl 1.3.26) xmerl_scan.erl:2133: :xmerl_scan.scan_element/12
(xmerl 1.3.26) xmerl_scan.erl:2605: :xmerl_scan.scan_content/11
(xmerl 1.3.26) xmerl_scan.erl:2133: :xmerl_scan.scan_element/12
(xmerl 1.3.26) xmerl_scan.erl:575: :xmerl_scan.scan_document/2
(xmerl 1.3.26) xmerl_scan.erl:291: :xmerl_scan.string/2

I’ll follow-up with these libraries directly.
Thanks again!

3 Likes
6

I’ve added a comment to this 2017 bug report

1 Like