I’m writing an application that needs to maintain many buffers that flush to a different location-partition combination in a rotating fashion. There are multiple locations with varying numbers of partitions.
In order to manage this, I’m planning on using an ETS table to maintain a set of records (one per location) that basically look like this:
{"location_key", max_partition, last_partition}
In the above example, max_partition
is the largest possible partition index of the given location and last_partition
is the last partition index of the given location that was written to.
Whenever one of the buffers flushes, I want to increment last_partition
using max_partition
as a threshold. I’m aware that there is a function that does almost exactly this but, as far as I can tell, I would have to provide the threshold when I make the call. Obviously, I could first fetch max_partition
and then use :ets.update_counter/4
but the trouble is that max_partition
can be updated (albeit rarely) so splitting the fetch and update operations opens the door for race conditions.
So, my question is: Is there a way to do all of this in one operation?
(As an aside, I’m also aware that this problem would be technically easier if I used a GenServer
, but this will be a very high traffic table so the performance of a GenServer
likely won’t cut it in production)