gyson
HyperLogLog is one of my favorite probabilistic algorithms. It’s a super space-efficient way for cardinality estimation. I recently used it at work via Presto and Spark and it works surprisingly well. Then I decided to make it from scratch to have better understanding how it actually works and chose lovely Elixir to implement it.
The rationale to make it Redis compatible is mostly because it’s an easy way to ensure that I have the correct implementation by checking that given same inputs, Redis and this module would yield the same output.
The Redis-compatible module used the same hash function (port it to Elixir), same estimation algorithm and same binary format. Therefore, you can use it to interact with Redis (v5) as well.
The repo is available at https://github.com/gyson/hll
P.S. During development, I found that binary pattern matching in Elixir/Erlang is pretty sweet for binary encoding and decoding !
Where Next?
Trending in Announcing
Other Trending Topics
Latest on Elixir Forum
Sponsor Spotlight
Supporting innovation across the BEAM ecosystem.
Categories:
Sub Categories:
Forums
Popular Tags
- #ecto
- #liveview
- #troubleshooting
- #learning-elixir
- #deployment
- #library
- #erlang
- #testing
- #genserver
- #mix
- #absinthe
- #remote-other
- #otp
- #plug
- #how-to-question
- #macros
- #postgres
- #channels
- #elixirconf
- #exunit
- #discussion
- #code-sync
- #javascript
- #podcasts
- #onsite
- #dialyzer
- #docker
- #authentication
- #umbrella
- #full-time-contract
- #podcasts-by-brainlid
- #ecto-query
- #elixir-ls
- #blog-post
- #phoenix_html
- #iex
- #graphql
- #genstage
- #ai
- #elixirconf-us
- #websockets
- #supervisor
- #advent-of-code
- #distillery
- #processes
- #api
- #forms
- #metaprogramming
- #hex
- #performance
Our Sponsors
Build Elixir applications with speed and confidence.
Supporting innovation across the BEAM ecosystem.
We build reliable cloud platforms for business-critical systems.
Catch errors, track performance, monitor hosts and more.
Error tracking, logs, uptime, and Just Enough APM. Built for developers.
Practical resources that improve the lives of professional developers.
Develop your skills with books, videos, and courses.
Bringing learning, work and compliance together since 1987.
The team behind Membrane, Popcorn, LiveDebugger. Available for hire.
Courses that'll move you from confusion to "Aha, now I get it!"
First 3 of 3 Posts
hauleth
Any particular reason for using Erlang records instead of Elixir structures?
gyson
It’s minor, using tuple/records would have slightly better performance (about 3~4% in my simple benchmarks) for this use case.
axelson
Records are used in the new MyXQL adapter for much the same reason (although the focus is on memory usage): Building a new MySQL adapter for Ecto, Part II: Encoding/Decoding « Plataformatec Blog