oliveiragahenrique

oliveiragahenrique

How to model a queue and consumer with GenServer?

I’m building a system which must have a queue and a kind of consumer, that will consume the events of the queue and remove them only if succeeds.

I’m thinking about model the queue and the consumer as different gen servers:

Queue process: Handle insert, get first event and delete operations
Consumer process: Handle consume operation (series of steps over the event data, which may fail)

Both processes have a 1:1 relationship, which means a specific queue has only one consumer and the consumer only consumes from one queue.

The consumer will call it self periodically, requesting to consume an event! This processes consist in a series of calls to the queue process, ultimately removing the event from the queue.

I’m currently thinking about some decisions:

1 - Would be useful to separate the registration of both process under two different Registry?
2 - Since consumer do not make sense without a queue, and the consumer can crash more easily, I was thinking if the Queue process could supervise the consumer process, thoughts on this?

How would you guys solve it, have some repository which implements something similar?

First Post!

kokolegorille

kokolegorille

Producer, Consumer… => GenStage

And other related tools, like Flow and Broadway. in case of error, it could be pushed into an error queue.

It’s nice to build something like this, but GenStage allow creation of pipeline of Producer, Consumer/Producer, Consumer with back pressure support.

Where Next?

Popular in Discussions Top

jswny
I would like to better understand what the advantages/disadvantages of umbrella applications are compared to structuring your app as as s...
New
blackode
Elixir Upgrading is so Simple in Ubuntu and It worked for me Ubuntu 16.04 git clone https://github.com/elixir-lang/elixir.git cd elixir...
New
Nvim
Anybody knows a comprehensive comparison of Django and Phoenix, thanks for the help. Where are they similar? Where do they differ the m...
New
thojanssens1
It would be nice to be able to define a redirect from one route to another from the router.ex file. E.g.: redirect "/", UserController, ...
New
AngeloChecked
What learn first? Rust or Elixir Hi Elixir community! I’m here because i want learn a new language. I’m a junior developer and mainly i ...
New
arcanemachine
https://nitter.net/josevalim/status/1744395345872683471 https://twitter.com/josevalim/status/1744395345872683471
New
jer
I’ve been using umbrellas for a while, and generally started off (on greenfield projects at least) by isolating subapps based on clearly ...
New
PragTob
Hey everyone, this has been on my mind for some time and I’d love your input on it! TLDR: I feel like maps are superioer for storing and...
New
wmnnd
The Go vs Elixir thread got me thinking: Would it be too hard to implement a simple mechanism for creating Go-style static app binaries f...
New
AstonJ
If so I (and hopefully others!) might have some tips for you :slight_smile: But first, please say which area you’re finding most challen...
New

Other popular topics Top

marius95
Hello everyone, I try to use an Javascript Event Handler in my root.html.leex file. Therefore I created a function in the app.js file: ...
New
malloryerik
Hi, this is for people who, like me, have had some friction using .html.heex templates in VSCode. The solution seems to be, in a hyphena...
New
fireproofsocks
Forgive me if this is obvious, but how does one delete a database record WITHOUT selecting it first? Ecto.Repo — Ecto v3.14.0 has exampl...
New
stefanchrobot
What’s the safe way to decode a JSON string into a struct? I want to avoid calling String.to_atom. Jason.decode can give me a map with st...
New
aesmail
Hello guys, I have finally made it. I created an admin interface for a framework. It’s been on my todo list for years and with the curre...
New
vrod
I am using the Starship cross-shell prompt – it seems pretty nice, but I get some errors: [WARN] - (starship::utils): Executing command ...
New
grych
Hi folks, Few months ago I have announced the proof-of-concept of the library to manipulate the browsers DOM objects directly from Elixi...
639 52341 488
New
nsuchy
Hi. I’ve noticed that Windows Powershell has it’s own IEX command and you cannot access Elixir’s IEX due to the conflict. This isn’t a cr...
New
WestKeys
Currently suffering from paralysis by [HTTP client] analysis. This is rather unusual in Elixirland as there tends to be consensus on the ...
New
sergio
Kind of like when jquery came out, it was super necessary. Existing drag and drop libraries have a bunch of baggage to support old browse...
New

We're in Beta

About us Mission Statement