How to write a caching server in Elixir

Welcome @highmountaintea to elixir.

Cachex can help you adding cache to your app.

One thing I did not understand from your question is you want to write as learning exercise or use it in a project.

GenServers and Agent can hold data in memory. They can be used to store state or data in memory. Best fit if you want to implement a Cache as a learning exercise.

Cachex can be used directly as cache in your project if you are looking to just use a cache library - like fetch some config from source and store in memory, refresh every n seconds, etc.

Hi Kartheek. It’s not meant for an exercise or an existing project. I am more interested in understanding how things are done in Elixir.

I did consider using Agent for this particular problem, but I am not sure if it would really satisfy my conditions as set above.

I took a look at Cachex you mentioned, but can you give an example of how Cachex would help prevent the situation I mentioned above? if person A asks a question, and person B asks the same question while the oracle is still pondering it, OracleServer would smartly avoid sending the same question to the oracle again, but informs both A and B when the answer comes back

You need to keep track of the question and who has asked it. When you have the answer, you send it to the references you kept track of. When a GenServer receives a call, you have the return address. Keep track of it.

awaiting = %{”Why?" => [pid<0.90.0>, pid<0.765.0>]}

Do you want all the questions & answers kept in memory, on disk or in a db?

You’re going to need a worker pool. Do you want to write that too or use a library?

Is this is the same thing you are looking for ?

As of v3, fallbacks changed quite significantly to provide the guarantee that only a single fallback will fire for a given key, even if more processes ask for the same key before the fallback is complete. The internal Courier service will queue these requests up, then resolve them all with the results retrieved by the first. This ensures that you don’t have stray processes calling for the same thing (which is especially bad if they’re talking to a database, etc.). You can think of this as a per-key queue at a high level, with a short circuit involved to avoid executing too often.

The new Courier service in Cachex v3 will actually queue the second and third calls to fire after the first one, rather than firing them all at once. What’s even better; the moment the first call resolves, the second and third will immediately resolve with the same results. This ensures that your fallback only fires a single time, regardless of the number of processes awaiting the result. This change in behaviour means that the code above would result in "key" having a single value of 1 as the second and third never fire. Although this results in a behaviour change above, it should basically never affect you in the same way as the code above is deliberately designed to highlight the changes.

Cachex takes care of these things automatically.

Hi cmo, thanks for your answer. So when the answer is available, what mechanism should I use to send the answer back to the requester(s)? I think I can send a plain message, but is there a more OTP way of doing it?

Also, if we are awaiting for answer inside the OracleServer, would it result in blocking, preventing others from asking questions simultaneously?

Hi kartheek, I am still trying to digest the fallback mechanism fully, but I think it is what I am looking for.

It’s nice to have a caching library that handles it automatically.

Forgot to answer your other questions…

Do you want all the questions & answers kept in memory, on disk or in a db? memory

You’re going to need a worker pool. Do you want to write that too or use a library? I would prefer to use a library

Ok well you’ll probably want to use ETS to cache your answers and there are a few worker pool libraries. Poolboy is a pretty common choice.

No problem just sending the messages.

Elixir In Action is super helpful in exposing you too all this sort of stuff.

Ha, this question did spring into my mind while I was reading Elixir In Action I remembered that each handle_call is blocking, so was trying to figure out how to design a non-blocking server that also handles caching.

Maybe I should re-read chapter 7 again, and some of it might become clear to me.

See how he keeps track of it in this Port example.