blubparadox
My elixir implementation of the 1 billion row challenge
Hi all. If you’ve looked on Twitter or YouTube you’ve seen the 1 billion row challenge, usually done in Java. I’ve written an Elixir version, feel free to have a look and see if you can improve on it.
https://github.com/rrcook/brc
Most Liked
garazdawi
I built an Erlang implementation that does 1B lines in 60 seconds on my slow laptop.
https://github.com/jesperes/erlang_1brc/pull/1
To improve it further, maybe one could use the counters module. So much to do, so little time…
stevensonmt
That post probably offers better insight than I can.
There’s also this blog post which offers the following explanation:
An ETS based approach
Other times, if the Agent doesn’t cut it for you, you might something faster. In these cases ETS might be a good option. The good thing about ETS is that it will always be faster because it doesn’t go through the Erlang Scheduler, furthermore it also supports concurrent reads and writes, which the Agent does not. However, it’s a bit more limited when you want to do atomic operations. Overall it’s very well suited for a simple shared key/value store, but if it’s better suited or not for your specific problem, that’s up to you.
Finally, the approach that immediately came to my mind was not using a cache at all but using Task.asyn_stream as described in this blog post
https://github.com/exercism/blog/blob/main/posts%2Fconcurrency-parallelism-in-elixir.md
tj0
Another implementation outside of the standard library if you’re interested.
Popular in RFCs
Other popular topics
Categories:
Sub Categories:
Forums
Popular Tags
- #ecto
- #liveview
- #troubleshooting
- #learning-elixir
- #deployment
- #library
- #erlang
- #testing
- #genserver
- #mix
- #absinthe
- #remote-other
- #otp
- #plug
- #how-to-question
- #macros
- #postgres
- #channels
- #elixirconf
- #exunit
- #discussion
- #code-sync
- #javascript
- #podcasts
- #onsite
- #dialyzer
- #docker
- #authentication
- #umbrella
- #full-time-contract
- #podcasts-by-brainlid
- #ecto-query
- #elixir-ls
- #phoenix_html
- #iex
- #blog-post
- #graphql
- #genstage
- #ai
- #websockets
- #supervisor
- #advent-of-code
- #elixirconf-us
- #distillery
- #processes
- #forms
- #api
- #metaprogramming
- #security
- #performance








