gdwoolbert3

gdwoolbert3

I’m writing an application that needs to maintain many buffers that flush to a different location-partition combination in a rotating fashion. There are multiple locations with varying numbers of partitions.

In order to manage this, I’m planning on using an ETS table to maintain a set of records (one per location) that basically look like this:

{"location_key", max_partition, last_partition}

In the above example, max_partition is the largest possible partition index of the given location and last_partition is the last partition index of the given location that was written to.

Whenever one of the buffers flushes, I want to increment last_partition using max_partition as a threshold. I’m aware that there is a function that does almost exactly this but, as far as I can tell, I would have to provide the threshold when I make the call. Obviously, I could first fetch max_partition and then use :ets.update_counter/4 but the trouble is that max_partition can be updated (albeit rarely) so splitting the fetch and update operations opens the door for race conditions.

So, my question is: Is there a way to do all of this in one operation?

(As an aside, I’m also aware that this problem would be technically easier if I used a GenServer, but this will be a very high traffic table so the performance of a GenServer likely won’t cut it in production)

Showing Posts 1 to 3

dimitarvp

dimitarvp

I would attempt :ets.select_replace but that only guarantees atomicity and isolation for a single object read + replace, not everything that matches.

Failing that, you can just use Cachex because it has what you need i.e. transactions.

cevado

cevado

as @dimitarvp pointed out you need a transaction. if you don’t want to add additional library, you can use either :global.trans/2 or :global.set_lock/1 and :global.del_lock/1.

lud

lud

A GenServer is fine if you do many reads and not so many writes.

A GenServer per-buffer could also work well, if you do not have too many buffers.

— All posts loaded —

Where Next? Top

Trending in Questions Top

RSP87
I’m working on a project that simulates the bumbl example in the programming phoenix book. It acts almost like an email client. We have a...
New
nseaSeb
Hello, I know there is an approach for handling lists that allows for optimized traversal, but I can’t recall the specific method (somet...
New
kpanic
Hi everyone, I am toying with the idea of building a “match maker” for giving personal help to people that wants to start coding. I sta...
New
brecabral
Documentation While reading the Scoped Routes section, I noticed that the documentation currently refers to a problem without explainin...
New
velrest
So my question is quite simple and i have found no conclusive answer on forum, google or AI. Should we use :erlang.float for Integer to ...
New
asweet-confluent
I recently noticed that Elixir’s Logger defaults its primary log level to :debug when no :logger, :level application configuration is pre...
New
apz
I’m new to elixir and just tried to install the elixirLS extension for VScode(ium) and it is throwing some errors that I would like help ...
New

Other Trending Topics Top

GenericJam
Edit: 2026 May 15 - This post is archived. Mob is alive!! Main docs: mob v0.7.11 — Documentation A bit of explanation for the slightly c...
New
JesseHerrick
Hey, I’m Jesse and I’m the main contributor behind Dexter, a full-featured, lightning-fast Elixir LSP optimized for large codebases. It s...
New
mudasobwa
I am happy to introduce the very α version of the new programming language compiled to BEAM. Welcome Cure. It has literally three kille...
New
marciok
Hi there! We created Gust: A task orchestrator inspired by Airflow. For those who have never heard about Aiflow, it’s a Python-based wor...
New
mhanberg
Hi everyone! The first release candidate for the Expert language server project is now available! We’ve published a press release detai...
New
jimsynz
Beam Bots (or just BB for short) is a framework for building fault-tolerant robotics applications in Elixir using familiar OTP patterns. ...
New

We're in Beta

About us Mission Statement

Options

Thread Display Mode




Thread Preview

Skip Thread Previews