Make atomic updates on ETS key

Few questions:

• What’s the reasoning for just using a single key? Usually it’s preferable to split the data over many keys in ets, which makes them easier to query.
• Is this simply an append and be fine storage? If so simply try to insert and let it fail if an url is already existing (would need refactoring away from using a list though).

I am not interested in querying anything. All I want this table to do is to save URLs whose requests failed. Then, when the time comes, the outside process will query the table for all URLs (always all) and I just returns them and clear the table.

So, 1 key is all I need.

I don;t quite understand. The :failed_urls key is supposed to have a list of urls. Are you suggesting I use a :bag ETS table instead of a set and then just have multiple URLs saved there?

Could you elaborate?

Try

insert_new(tab, {url,url})

for each url - later foldl over the entire table.

I was actually thinking about using http://erlang.org/doc/man/ets.html#take-2 to get everything with the same key and remove it in one go

:ets.delete_all_objects(tab)

delete_all_objects doesn’t actually return them, which is something I want in addition to deleting them

Then is really going to be a :bag

tab = :ets.new(name, [:bag])
:ets.insert(tab, {:failed, url1})
:ets.insert(tab, {:failed, url2})
failed = :ets.take(tab, :failed) |> Keyword.values()
# [url1, url2]

:bag comes with the implementation overhead of checking for duplicates on insert; so unless you explicitly need to prevent duplicate full records, you should use :duplicate_bag over :bag .

i.e. may be worth dealing with duplicates after take/2

Could you elaborate ?

It might be better for performance to us :duplicate_bag, which is mentioned to be faster on inserts, and deduplicate the urls after using take/2.