Design microservice to handle significant payload

I want to create simple microservice. It intended to be called by another elixir process. So it could be module with single function like:

defmodule MicroMod do

def get_info(key) do

end

end

The get_info gets called by many clients with key param, the MicroMod should maintain some kind of cache,
so get_info might reply immediately, but might block for some time (getting info from another service).

For example MicroMod cache looks like { “a” => ok } and 2 clients come almost simultaneously and call get_info, but client 1 was the first:

client1: MicroMod.get_info(“b”)
client2: MicroMod.get_info(“a”)

So here is the situation: client1 waits 2 seconds for information on “b” and that’s OK, because “b” is not in cache and should be obtained from slow surce, but client2 forced to wait 2 seconds because gen_server serves only one client at time, so client 2 waits for client 1 to be served, but this is totally wrong! Client2 should get immediate reply for his cached key here! Any ideas on solving this ? Or i suspect i just completely don’t understand something here.

1 Like

You are completely correct, a single GenServer process is a bottleneck. It is generally recommended to use ETS for use cases like this. Each request to your service should spawn a new process that gets the answer from ETS or creates it and puts it there. There are a couple of caching libraries that implement these primitives for you, so you can specify just a key, function to call and a time for the value to live.

2 Likes

The use-case as you’ve laid it out here sounds like a perfect fit for cachex: https://github.com/whitfin/cachex

4 Likes