simonbowen

simonbowen

Ordering issue with erlang queue and an ETS table

I have a system which I need to feed data into and maintain. Currently I have a GenServer that communicates with an ETS table and inserts items into the table. As part of this I have a FIFO erlang queue, when an item is added to the ETS table I am pushing it to the queue to be consumed from elsewhere.

For some reason the ordering of the items in the ETS and Queue are different and I cannot work out why. This is the code I am using to push the item into the ETS table and Queue.

with :ok <- Queue.add(job), true <- :ets.insert(table, {id, job}) do
    PubSub.broadcast!(BrokerEx.PubSub, @topic, {:added, job})
end

When I dump the contents of the table and the queue they are not the same. The Queue in this context is a simple GenServer that manipulates Erlangs built in queue module.

Where am I going wrong?

Marked As Solved

simonbowen

simonbowen

I’ve worked it out. Using table = :ets.new("table", [:set, :protected]) creates an ETS table with the type of set and there is no guarantee of the order with this table type.

To be honest I think I need to use a RDBMS at this point anyway since I need to have some persistence for this data anyway.

Also Liked

dimitarvp

dimitarvp

Yep, was about to say “why not use a database”? You can also use timeseries if needed – not so hard to install the timescale Postgres extension. Or you can just queue things up in Redis and have background workers pull them away from there and store them in a regular DB.

Where Next?

Popular in Questions Top

albydarned
Hello all! I am typing this post from my new MacBook Pro with the M1 chip. I’m loving it so far, and will probably use it as my daily dr...
New
skosch
To my knowledge, put_in, Map.update etc. all have the one limitation of not automatically creating intermediate keys when needed (for exa...
New
electic
Hi, I am new to Elixir. I am trying to use the DateTime component to insert a date into MySQL however the there seems to be no way to fo...
New
tduccuong
Hi, is there any work on GUI with Elixir, that is similar to Electron/Javascript? My idea is to bundle Phoenix and BEAM into a single se...
New
vegabook
I’m brand new to Phoenix and I have stripped one of the demo applications to the bone. I just want to get an svg up on the screen. Here i...
New
SoCreat
i’m a new one to elixir which editor can i use vs code? or atom? Thanks! :smiley:
New
JDanielMartinez
Hi! May someone helps me, please! I have two apps into an umbrella project: the first one is Database, which manages queries, and the se...
New
dblack
I’ve got an issue with an app and I’ve no idea of how to troubleshoot it. I’m hoping someone here might have seen something similar. I p...
New
WestKeys
Currently suffering from paralysis by [HTTP client] analysis. This is rather unusual in Elixirland as there tends to be consensus on the ...
New
vonH
In asking this question I am more interested about the expressiveness of the language itself and less concerned about the availability of...
New

Other popular topics Top

aadeshere1
I have a another noob question about loop. Since elixir is immutable, while loop is not directly possible. total = 10 while total != 0 ...
New
sorentwo
Hello! tl;dr Announcing Oban, an Ecto based job processing library with a focus on reliability and historical observability. After spen...
985 42920 311
New
senggen
Erlang/OTP 25 [erts-13.2.2] [source] [64-bit] [smp:8:8] [ds:8:8:10] [async-threads:1] 15:22:35.803 [error] gen_event {lager_file_backend...
New
siddhant3030
Hi, I have to write a raw query for one of my project. But till now I have used ecto queries and don’t have much experience writing raw ...
New
ovidiubadita
Hey all, I discovered Elixir and I love it. I always wanted to learn a functional programming and I intended to go for Haskell, but afte...
New
johnnyicon
Hi all, I’ve just started learning Elixir and Phoenix Framework, so please pardon my n00bness at this stage. I’m trying to use Postgres...
New
vonH
When I run the Plug and I recompile I wind up having to use Ctrl C to quit iex and start again. Witht the help of rlwrap I can use the cu...
New
freewebwithme
Using vs code and installed ElixirLS: support and debugger. And I got an error popped up on start up says Failed to run ‘elixir’ comma...
New
boundedvariable
I am going through the kafka architecture. All the features what the kafka is providing are already in Erlang. I would like hear your opi...
New
WestKeys
Currently suffering from paralysis by [HTTP client] analysis. This is rather unusual in Elixirland as there tends to be consensus on the ...
New

We're in Beta

About us Mission Statement