samkit

samkit

Acquire locks in ets table row in order they were requested

I have n number of identical rabbitmq stream consumers with prefetch count set to 1. All the messages have a tag, lets suppose user_id in this case. The messages with the same user_id tag must be processed in order they arrive in the stream.

The consumer fetches the state corresponding to the tag from ets, acquires a lock ,does processing, update the row and release the lock. Now, lets suppose consumer 1,2,3 all have messages with the same tag while consumer 2 and 3 are trying to acquire the lock in a loop and consumer 1 having the lock. How do I ensure that consumer 2 gets the lock first/message 2 is processed first after consumer 1 has released the lock.

Thanks for reading:)

My current implemetation has a single consumer consuming all the messages with the same tag, therefore order is maintained automatically, but in this case I need to have as many consumers as the number of different tags/users on the platform which doesnt scale out very well.

Most Liked

al2o3cr

al2o3cr

This talk walks through a very similar problem statement:

Short short version: it presents an algorithm for routing RabbitMQ messages so that ones that need consistent ordering are always processed by the same consumer so (for instance) the ones with user_id = 1 are seen in submission order.

Last Post!

samkit

samkit

Hi Matt. In our current implementation, we have this setup already where the messages with same tag are consumed by a single consumer. However, in that case we have to have as many consumer as users on out platform.
We are trying to design a generic consumer and handle sequence using ets locks. So, can workers trying to receive lock acquire in the order of demand made for the lock.

Where Next?

Popular in Questions Top

ovidiubadita
Hey all, I discovered Elixir and I love it. I always wanted to learn a functional programming and I intended to go for Haskell, but afte...
New
lanycrost
Hi everyone! I need implement if…else if…else condition from my elixir code, and anymore of this control flow structures not work proper...
New
mcarvalho
What is the difference between System.get_env and Application.get_env? For example, what are best practices to use one versus another.
New
joeerl
Hello again - after a longish gap I’ve decided I really must dig into Elixir and see what’s been happening here - so I have a few questio...
New
sen
Hi All, I set a environment variables in dev.exs , like below code. when i start server, how can i set the ${enable} value? thanks. d...
New
SoCreat
i’m a new one to elixir which editor can i use vs code? or atom? Thanks! :smiley:
New
dblack
I’ve got an issue with an app and I’ve no idea of how to troubleshoot it. I’m hoping someone here might have seen something similar. I p...
New

Other popular topics Top

baxterw3b
Hi guys, i’m new in the Elixir world, and i have to say, that i love it! i’m having some problem to understand anonymous functions with ...
New
Qqwy
Original source of discussion: This topic on the Pragmatic Programmers’ Functional Web Development with Elixir, OTP, and Phoenix forum. ...
New
ashish173
I am using Ecto timestamps with postgres, I can see the timestamps() use the :naive_dateime but for my use case I wanted to store the ti...
New
joeerl
Hello again - after a longish gap I’ve decided I really must dig into Elixir and see what’s been happening here - so I have a few questio...
New
msaraiva
Surface is an experimental library built on top of Phoenix LiveView and its new LiveComponent API that aims to provide a more declarative...
564 44139 214
New
jason.o
In the code below, if the create action is not set to accept “extra_key” as an input, it errors out with a message shown above. Is there ...
New

Latest on Elixir Forum

Elixir Forum

We're in Beta

About us Mission Statement