samkit

samkit

Acquire locks in ets table row in order they were requested

I have n number of identical rabbitmq stream consumers with prefetch count set to 1. All the messages have a tag, lets suppose user_id in this case. The messages with the same user_id tag must be processed in order they arrive in the stream.

The consumer fetches the state corresponding to the tag from ets, acquires a lock ,does processing, update the row and release the lock. Now, lets suppose consumer 1,2,3 all have messages with the same tag while consumer 2 and 3 are trying to acquire the lock in a loop and consumer 1 having the lock. How do I ensure that consumer 2 gets the lock first/message 2 is processed first after consumer 1 has released the lock.

Thanks for reading:)

My current implemetation has a single consumer consuming all the messages with the same tag, therefore order is maintained automatically, but in this case I need to have as many consumers as the number of different tags/users on the platform which doesnt scale out very well.

Most Liked

al2o3cr

al2o3cr

This talk walks through a very similar problem statement:

Short short version: it presents an algorithm for routing RabbitMQ messages so that ones that need consistent ordering are always processed by the same consumer so (for instance) the ones with user_id = 1 are seen in submission order.

Last Post!

samkit

samkit

Hi Matt. In our current implementation, we have this setup already where the messages with same tag are consumed by a single consumer. However, in that case we have to have as many consumer as users on out platform.
We are trying to design a generic consumer and handle sequence using ets locks. So, can workers trying to receive lock acquire in the order of demand made for the lock.

Where Next?

Popular in Questions Top

hariharasudhan94
I would like to know what is the best IDE for elixir development?
New
Darmani72
If I have a post route which an argument: post /my_post_route/:my_param1, MyController.my_post_handler How would get the post params ...
New
greenz1
I have a phoenix application from which a user can download multiple(5-6) files of size 1MB. I couldn’t find anything related to sending ...
New
siddhant3030
Hi, I have to write a raw query for one of my project. But till now I have used ecto queries and don’t have much experience writing raw ...
New
fayddelight
I tried installing elixir 1.11.2 erlang 23.3.4 via asdf in my zsh shell. Enabled the versions locally and globally. When I list them ...
New
JorisKok
I have a server on AWS, and was running a load test using artillery. When looking at the Phoenix dashboard I see the Ports going to 100% ...
New
senggen
Erlang/OTP 25 [erts-13.2.2] [source] [64-bit] [smp:8:8] [ds:8:8:10] [async-threads:1] 15:22:35.803 [error] gen_event {lager_file_backend...
New

Other popular topics Top

Qqwy
Update: How to use the Blogs & Podcasts section You can post links to your blog posts or podcasts either in one of the Official Blog...
3271 130286 1222
New
JeremM34
Hello, how can I check the Phoenix version ? Thanks !
New
jononomo
For some reason my phoenix channels are working for me in my local dev environment, but as soon as I deploy via Docker, I get a 403 error...
New
dokuzbir
I want to highlight html closing tags when i click a html tag. That works in .html files but doesnt work for html.eex templates. How can...
New
Patoshizzle
After calling mix ecto.create I get this error: 17:00:32.162 [error] GenServer #PID<0.412.0> terminating ** (Postgrex.Error) FATAL...
New
jason.o
In the code below, if the create action is not set to accept “extra_key” as an input, it errors out with a message shown above. Is there ...
New

We're in Beta

About us Mission Statement