OvermindDL1

Erlang 21.2 erts-10.2 released

A few good fixes and enhancements, SSL added more methods and is faster, socket polling is faster, however 2 big things that I like!

New counters and atomics modules supplies access to highly efficient operations on mutable fixed word sized variables.

The new :atomics module wraps the low level hardware atomic instructions without any locking, it’s use-cases are limited but EXTREMELY useful for where they are useful. I find it odd how the array part is 1-indexed though, erlang has a lot of mixed 0-index and 1-index code, and 1-indexed means the math is harder… blah…

The new :counters module uses the new :atomics module to do ETS-style counter handling but using hardware atomic instructions so there is no locking or cost. Same thing with the really weird non-mathematical 1-indexed arrays though.

New module persistent_term!. Lookups are in constant time! No copying the terms!

You know the global module that discord put out that recompiles a module to bake terms into it for super-fast referencing and no-copies and so forth? That’s basically the new erlang persistent_term library! Here’s a use session in Elixir!

iex(1)> :persistent_term.info()                        
%{count: 0, memory: 152}
iex(2)> :persistent_term.put(:blah, 42)
:ok
iex(3)> :persistent_term.info()        
%{count: 1, memory: 192}
iex(4)> :persistent_term.get(:blah)
42
iex(5)> :persistent_term.get()     
[blah: 42]
iex(6)> :persistent_term.put(:blorp, make_ref())
:ok
iex(7)> :persistent_term.get()                  
[blah: 42, blorp: #Reference<0.3525248772.1957167107.195526>]
iex(8)> :persistent_term.erase(:blah)
true
iex(9)> :persistent_term.get()                  
[blorp: #Reference<0.3525248772.1957167107.195526>]
iex(10)> :persistent_term.info()
%{count: 1, memory: 216}

So it’s slow to put but crazy-fast to get, reference, and pass around the values stored within! In general you shouldn’t use this unless you know what you are doing as there is no garbage collection, should generally be used for global settings and data, etc…

14 comments

#erlang #erlang-news

24 3803 14

2019-09-18 22:12:34 UTC

Most Liked

michalmuskala

Changing anything in persistent_term means running a GC over all the processes in the system and copying all of the data in persistent_term. It can get very expensive on bigger systems.

Post #6

michalmuskala

One thing to ask is, if it would be actually beneficial. Binaries (and I assume your entries are just plain binaries) are stored off-heap, so they are not copied when retrieved from ETS anyway. This removes the biggest advantage of persistent_term over plain old ETS.

Post #9

OvermindDL1

Think of it this way: If it’s something you would normally compile ‘into’ a models code for efficiency then use it, else you probably want ETS (or mnesia to keep it serialized to disk and hold it in memory on load).

So for a blog I’d personally use either flat-files on disk or store it in mnesia with disk and memory copies, I’m not sure I’d use this for that, though your data is static enough that it ‘should’ be fine.

Just remember, :persistent_term has a max storage size of 1 gig unless you set the _MIscs 2048 or so option (that one would raise it to 2 gigs, the value is in megabytes).

When a new value is put then it is visible everywhere immediately on the node because when an old value is replaced or erased then a global garbage collection is run, which can pause things, that collection will replace any links to the global data in all processes to a local copy of the data instead.

Just remember that accessing data in :persistant_term is way fast, but changing data in any way is very slow, and in some cases it can be really really slow. But it’s still faster than recompiling a module to ‘intern’ the terms (which would destroy linked processes if it got updated, so :persistent_term is definitely better there).