Correct Way to Use Persistent Term

Questions / Help
1

Have been trying to figure out the correct way to code and use a persistent term, and am stuck.

  1. How do I assign a variable to a persistent term that I can access outside the module it was created in?
  2. How would I write the following code?

def build_store do

#Setup Persistent Term variable here

#Stream CSV File. I’m ok with this part using File.stream!() and parsing using split
|> Stream.map(fn x -> #Add to persistent term here# end) #This bit has me stumped

end

#Once term is created
#How do I read it here from another module?

Any assistance will be greatly appreciated :grinning:

1 Like
2

:persistent_term is VM-wide storage - so accessible from any module. This can create a bit of a challenge for naming so I typically use {module_name, key} as keys for persistent term.

The other consideration is balancing between a persistent storage and an amount of copying. Therefore I find myself using :persistent term when I can flatten the structure somewhat. For example, instead of storing a full map, I store each key of the map (and its value) in :persistent_term.

So to your example: Any module can access :persistent_term so stream and store to your hearts content.

5 Likes
3

Thx Kip.

Do you perhaps have a code sample of the correct use of persistent term? I have searched high and low and been unsuccessful in finding one.

4

I don’t seem to have a good example immediately to hand. I do have an example, but arguably its not a good one. Anyway, you’re welcome to whatever value it may contribute:

Really, its a very niche, almost experimental, use case to emit a logger warning only once. Probably not a good example, but the only one I have to hand at the moment.

2 Likes
5

Thx Kip,

Those examples are actually quite helpful. Please confirm if my understanding is correct. You basically use it as follows:

  1. Create a unique key with the structure: modulename_key e.g mynamestore_firstname, mynamestore_lastname, mypricestore_socks

  2. Save your value as:
    `
    :persistent_term.put(“mynamestore_firstname”, “Mark”)
    :persistent_term.put(“mynamestore_lastname”, “Johnson”)
    :persistent_term.put(“mypricestore_socks”, “25.00”)
    etc…

  3. Then to access you use:

first_name = :persistent_term.get(“mynamestore_firstname”)
last_name = :persistent_term.get(“mynamestore_lastname”)
sock_price = first_name = :persistent_term.get(“mypricestore_socks”)

Is that correct?

6

Be sure to check out the performance implications of using persistent_term: http://erlang.org/doc/man/persistent_term.html#description

Namely updating and deleting terms may result in a global garbage collection pass that may or may not be acceptable on your use case.

4 Likes
7

Thx Nicd,

Yes, that is a very important consideration. The use case is a global lookup table that is only read once from a csv file and never updated. I believe it is ok in this use case. Is that correct?

3 Likes
8

Yes AFAIK that would be a good use case for persistent_term.

4 Likes
9

I have tried this code and it is not working:

defmodule LookupTable do

def build_lookup do

load_path = Path.join([".", "data", "lookup.csv"])


File.stream!(load_path,[:read, :utf8]) #Read File
|> Stream.map(&String.trim(&1)) #Trim Each Element
|> Stream.map(&String.split(&1, [","])) #Split at Delimiters
|> Stream.map(fn n -> :persistent_term.put(hd(n),tl(n)) end)
|> Enum.map(&IO.inspect(&1))

:persisent_term.get("global_name")

end

end

I get the following error:

** (UndefinedFunctionError) function :persisent_term.get/1
is undefined (module :persisent_term is not available)
:persisent_term.get(“global_name”)

10

:persistent_term is only available on OTP 21.2 and later so I suspect you are on an older OTP release.

2 Likes
11

Thx Kip,

I did upgrade both Erlang and Elixir to the latest release. :persistent_term does work in iex.

12

You have a typo in your code :slight_smile:
You have persisent_term not persistent_term on the last line.

2 Likes
13

Thx, now I really feel daft :crazy_face:

1 Like
14

About this

reading the :persistent_term documentation they say

When a persistent term is updated or deleted, a global garbage collection pass is run to scan all processes for the deleted term, and to copy it into each process that still uses it.

scan all processes: does it mean all the processes running in the VM, or just the processes that previously were using the delete/updated term?
if I have 100,000 processes and just few of them are dealing with persistent term?

15

As the docs said, all processes needs to be scanned to see which of them have a reference to the PT. There is no list of processes which have a reference, as the reference from the process might have been collected already via a regular GC, or the process might have sent copies of the reference to another process.

Keeping track of this would add additional complexity.

5 Likes
16

Cowboy’s documentation is also mentioned about simple usage of persistent_term.

See https://ninenines.eu/docs/en/cowboy/2.7/guide/routing/ |> Using persistent_term