ETS Cache multiple tables from Ecto

Questions / Help
, ,
1

Hi
I am using ETS to cached a database schema from postegress using ecto
here are those examples:

table = :ets.new(:cache_name,[:set, :protected])

and include those registry:

:ets.insert(table,{:first_table,{1}})
:ets.insert(table,{:first_table,{5}})
:ets.insert(table,{:second_table,{1}})

but the second one replace the first one, for that reason i concat the table name and the id to get a unique key :ets.insert(table,{:first_table1,{1}}) for these registry, but at the moment that i want the first registry for the first table i have a problem because i contain the same key for the second and it retreive two registry:

:ets.match_object(table,{:“_”,{1}})

how can i specify to ETS that if the key contains the table_name retreive these registry?

2

First, I’d be absolutely sure that cacheing this is good, postgresql is very very fast and you have to worry about concurrency issues and staleness with caches.

Second, I’d recommend using CacheX or so, you can set up a resolver so you ask cachex for something, if it does not exist then it goes to, say, your DB to fetch it and cache it. It has timeout support, janitors, etc… All built on ETS.

Third, if you want to key something by a table with an identifier, than wrap those up in a tuple in the key position, so like:

:ets.insert(table, {{:first_table,1}, :other_data})
:ets.insert(table, {{:first_table,5}, :other_data})
:ets.insert(table, {{:second_table,1}, :other_data})
1 Like
3

Thanks.

4

Does CacheX handle deduplicating requests if it doesn’t have what you are looking for? I.e. - 50 processes ask for the same missing thing, CacheX gets it once?

5

Looking at the code, it doesn’t seem to de-dup-fetch-on-cache-miss. This is understandable imho as it is unclear from the library’s limited view into your application when the fallback should be run only once, or called on each and every miss no matter what. Buuuuut … it does look like you could serialize this yourself. There are various ways the serialization could be achieved without much effort through the semicreative use of processes and messages :slight_smile:

2 Likes
6

Answering my own question, sort of…ConCache does it…

# Returns existing value, or calls function and stores the result.
# If many processes simultaneously invoke this function for the same key, the function will be
# executed only once, with all others reading the value from cache.
ConCache.get_or_store(:my_cache, key, fn() ->
  initial_value
end)

Not sure about CacheX though.

7

Yep it can, you can even set a default resolver for a cache so you can just always ‘get’ the value from the cache and if it does not exist or too old or so then it uses the default resolver to get it, cache it, and return it to you. It is quite convenient.

8

But it does limit that to a single executing resolver?

9

It incurs no built-in limits of the sort. If you use the direct interface then it will be as multi-threaded as it can possibly be. If you use CacheX Transactions (basically a ‘get’ with also saying what key’s you want to ‘lock’ during the transaction) then you can prevent concurrent access and have a single one done always. In general CacheX is designed to be as fast as possible, calling out to no other processes, direct ETS access, secondary janitor processes, etc… etc…

EDIT: And of course you can always have the fallback/resolver call out to a genserver or something too, that way no changes to user code are necessary.

1 Like