highmountaintea

How to write a caching server in Elixir

There is an oracle on the moon, who can give profound answers to all questions. The oracle is all knowing, but requires a minute or two to ponder each question before answering. Luckily he has 4 brains, so he can ponder 4 questions simultaneously. Unfortunately the oracle has very bad memory, so if you ask the same question over and over again, he will have to think about it every time.

How do we write an OracleServer in Elixir that can query the oracle for answers? It needs to do the following:

allow multiple people to ask questions at once, because the oracle has 4 brains
cache all the answers, to avoid asking repeat questions
if person A asks a question, and person B asks the same question while the oracle is still pondering it, OracleServer would smartly avoid sending the same question to the oracle again, but informs both A and B when the answer comes back
the answer is not always 42

Obviously it’s a thinly disguised way to describe a well known problem. I am familiar with solutions in other languages, but am having trouble figuring out how to do it in Elixir.

17 comments

#cache #server #memoize

1 2180 17

2022-01-28 17:18:23 UTC

Marked As Solved

kartheek

Cachex has get, put, fetch, clear, etc.

get returns value if it is present in cache, nil if its a cache miss
fetch executes fallback function on cache miss.

For example - lets have a fallback function which returns same value after delay of 5 seconds.

iex(1)> import Cachex.Spec
iex(2)> Cachex.start_link(:my_cache, [ fallback: fallback(default: fn x -> :timer.sleep(5000)
...(2)> x end) ]) 
iex(3)> Cachex.get(:my_cache, 1) #get will return nil as there is no entry
{:ok, nil}

fetch will invoke fallback function when there is a miss - fallback function is executed and value is put into the cache after delay of 5 seconds.

iex(4)> Cachex.fetch(:my_cache, 1) # 5 sec delay due to timer.sleep in fallback function
{:commit, 1}
iex(5)> Cachex.get(:my_cache, 1) # get will return value as it exists in cache
{:ok, 1}

Now fetch will return value from cache without executing fallback function

iex(6)> Cachex.fetch(:my_cache, 1) # returns immediately, fallback not invoked
{:ok, 1}

Post #11

Also Liked

kartheek

Is this is the same thing you are looking for ?

As of v3, fallbacks changed quite significantly to provide the guarantee that only a single fallback will fire for a given key, even if more processes ask for the same key before the fallback is complete. The internal Courier service will queue these requests up, then resolve them all with the results retrieved by the first. This ensures that you don’t have stray processes calling for the same thing (which is especially bad if they’re talking to a database, etc.). You can think of this as a per-key queue at a high level, with a short circuit involved to avoid executing too often.

The new Courier service in Cachex v3 will actually queue the second and third calls to fire after the first one, rather than firing them all at once. What’s even better; the moment the first call resolves, the second and third will immediately resolve with the same results. This ensures that your fallback only fires a single time, regardless of the number of processes awaiting the result. This change in behaviour means that the code above would result in "key" having a single value of 1 as the second and third never fire. Although this results in a behaviour change above, it should basically never affect you in the same way as the code above is deliberately designed to highlight the changes.

Cachex takes care of these things automatically.

Post #4

axelson

Scenic Core Team

Ah, I meant using the pool in combination with Cachex and not in the worker processes. Although I now realize that was not very clear. The reason that using a pool is important with Cachex for your use-case is that If you’re using the fallback function with Cachex then you could have more than 5 fallback functions running at the same time if you’re trying to fetch more than 5 keys at the same time. By using a pool you can explicitly limit the number of fallback functions that are currently accessing the oracle on the moon.

Post #16

cmo

You need to keep track of the question and who has asked it. When you have the answer, you send it to the references you kept track of. When a GenServer receives a call, you have the return address. Keep track of it.

awaiting = %{”Why?" => [pid<0.90.0>, pid<0.765.0>]}

Do you want all the questions & answers kept in memory, on disk or in a db?

You’re going to need a worker pool. Do you want to write that too or use a library?

Post #3