Alex66

Alex66

Ultra_log_log 0.2 - UltraLogLog distinct counting, now with a lock-free concurrent insert path

ultra_log_log is an Elixir implementation of UltraLogLog (Ertl, VLDB 2024), a probabilistic data structure for approximate distinct counting — a more space-efficient successor to HyperLogLog. Same constant memory, constant-time inserts, and mergeable-as-a-CRDT property; 24–28% less memory at the same accuracy.

0.2 adds a lock-free concurrent insert path. UltraLogLog.Concurrent is backed by :atomics with a CAS loop — no GenServer, no lock, safe to insert into from any process or scheduler. snapshot/1 converts it back to the immutable form for estimation, merge, or serialization.

Example:

{:ok, c} = UltraLogLog.Concurrent.new(precision: 12)

1..1_000
|> Enum.chunk_every(100)
|> Enum.map(fn chunk ->
  Task.async(fn -> Enum.each(chunk, &UltraLogLog.Concurrent.add(c, &1)) end)
end)
|> Task.await_many(:infinity)

sketch = UltraLogLog.Concurrent.snapshot(c)
{:ok, count} = UltraLogLog.cardinality(sketch)

Three estimators are available (FGRA, MLE, martingale), all
validated bit-for-bit against the Hash4j Java reference.

Hex: ultra_log_log | Hex
Docs: UltraLogLog v0.2.0 — Documentation
GitHub:

https://github.com/thatsme/ultra_log_log

Full design notes, benchmarks, and validation methodology are in
the README and the docs. Feedback welcome.

Where Next?

Popular in Announcing Top

mathieuprog
Hello :waving_hand: Allow me to introduce you to Tz, an alternative time zone database support to Tzdata. Why another library? First a...
New
jakub-zawislak
Hi everyone, I’m coming from the Symfony (PHP) framework. I like Phoenix, but it has a one thing that was build much better in the Symfo...
New
brainlid
LangChain is short for Language Chain. An LLM, or Large Language Model, is the “Language” part. This library makes it easier for Elixir a...
New
zorbash
I created Kitto a framework for dashboards inspired by Dashing. The distributed characteristics of Elixir and the low memory footprint...
New
type1fool
WebAuthnLiveComponent WebAuthnComponents See this post about renaming the package. Passwordless authentication for Phoenix LiveView app...
New
Crowdhailer
Raxx is an alternative to Plug and is inspired by projects such as Rack(Ruby) and Ring(Clojure). 1.0-rc.1 is now available. To use it re...
New
KronicDeth
Elixir plugin for JetBrain’s IntelliJ Platform (including Rubymine) This is a plugin that adds support for Elixir to JetBrains IntelliJ...
289 36352 110
New
michalmuskala
Another small library today. PersistentEts Hex: persistent_ets | Hex GitHub: GitHub - michalmuskala/persistent_ets · GitHub Ets table ...
New
archan937
It is a well-know topic within the Elixir community: “To mock or not to mock? :)” Every alchemist probably has his / her own opinion con...
New
scohen
Lexical Lexical is a next-generation language server for the Elixir programming language. Features Context aware code completion As-you...
New

Other popular topics Top

New
sorentwo
Hello! tl;dr Announcing Oban, an Ecto based job processing library with a focus on reliability and historical observability. After spen...
985 43487 311
New
pmjoe
I have a relationship of love and hate with Elixir. Lots of things are just absolutely right, but there are some things that are kind of ...
New
jay1
Why is it that the mnesia database isn’t the most preferred database for use in Elixir/Phoenix?
New
vonH
When I run the Plug and I recompile I wind up having to use Ctrl C to quit iex and start again. Witht the help of rlwrap I can use the cu...
New
grych
Hi folks, Few months ago I have announced the proof-of-concept of the library to manipulate the browsers DOM objects directly from Elixi...
639 52673 488
New
bsollish-terakeet
Credo is smart enough to check for (something like) this: assert length(the_list) == 0 with this response: Checking if an enum is empt...
New
jononomo
For some reason my phoenix channels are working for me in my local dev environment, but as soon as I deploy via Docker, I get a 403 error...
New
dokuzbir
I want to highlight html closing tags when i click a html tag. That works in .html files but doesnt work for html.eex templates. How can...
New
svb
Hi! Currently I want to submit a form by pressing the Enter key. However, since my input field is of type “textarea” this is just adds a...
New

Latest on Elixir Forum

Elixir Forum

We're in Beta

About us Mission Statement