dominicletz
Ex_sha3: Pure Elixir implementation of sha3 AND keccak-1600-f
Implementing ex_sha3 was an experiment of a pure Elixir version of the fips-202 sha3 and original keccak algorithms. ex_sha3 works platform independent without any nif wrapper, requires no c compilation. Also, this is the first library I’m aware of that exposes both algorithms, the original keccak hash as used in Ethereum, and the sha3 hash that got standardized. They are slightly different in padding and so return different hash results. Most implementations only do one or the other.
Size wise I’m pretty happy about this port. The implementation is in a single .ex file of 232 lines of code. Derived from the tiny-keccak implementation used by nim, exposing sha3, keccak and shake independently.
Compare exsha3/lib/ex_sha3.ex at master · dominicletz/exsha3 · GitHub vs. nim-keccak-tiny/keccak_tiny/keccak-tiny.c at master · status-im/nim-keccak-tiny · GitHub
Unfortunately, the performance is abysmal compared to existing nif based implementations:
> mix run benchmark.exs
Operating System: Linux
*snip*
Comparison:
short nif_sha3_256 198.18 K
long nif_sha3_256 2.13 K - 93.24x slower +465.43 μs
short ex_sha3_256 0.102 K - 1946.84x slower +9818.31 μs
long ex_sha3_256 0.00069 K - 285946.87x slower +1442824.24 μs
Haven’t yet looked into optimizing the performance as I was just in need of a short correct implementation, but would be happy to entertain anything to speed it up. The 2000x performance difference is a bit devastating I have to say.
Source: GitHub - dominicletz/exsha3: Pure Elixir implementation of Sha3 and the original Keccak1600-f · GitHub
Hex.pm: ex_sha3 | Hex
Docs: ExSha3 — ex_sha3 v0.1.5
Most Liked
benwilson512
It’s worth noting that due to last call elimination, tight recursion can be basically the same thing as mutation. For example:
def sum([], acc), do: acc
def sum([h | tail], acc), do: sum(tail, h + acc)
The acc variable will be essentially mutated in place. Similar things can be done for binaries, so I expect performance can be improved by several orders of magnitude here.
dominicletz
Thanks @Schultzer’s pull request and a faster rol() implementation we’ve got a new version now, that is still pure Elixir but nearly 10x faster… That said it’s still much slower than the nif:
##### With input long string #####
Comparison:
nif_sha3_256 2243.02
ex_sha3_256 9.36 - 239.60x slower +106.37 ms
##### With input short string #####
Comparison:
nif_sha3_256 215033.52
ex_sha3_256 870.25 - 247.09x slower +1.14 ms
blatyo
There are some algorithms, where the only known efficient implementations rely on in place mutation. I wouldn’t be surprised if this was one of them. If this is one of those, it’s likely you’ll always be better off using a NIF, which is ok. The language makes that tradeoff so we can get other things much more trivially, like concurrency and isolation.
There are some mutation API’s that in some situations might allow you to eek out better performance. Such as ETS and the process dictionary.
Popular in Announcing
Other popular topics
Latest on Elixir Forum
Sponsor Spotlight
Build Elixir applications with speed and confidence.
Categories:
Sub Categories:
Forums
Popular Tags
- #ecto
- #liveview
- #troubleshooting
- #learning-elixir
- #deployment
- #library
- #erlang
- #testing
- #genserver
- #mix
- #absinthe
- #remote-other
- #otp
- #plug
- #how-to-question
- #macros
- #postgres
- #channels
- #elixirconf
- #exunit
- #discussion
- #code-sync
- #javascript
- #podcasts
- #onsite
- #dialyzer
- #docker
- #authentication
- #umbrella
- #full-time-contract
- #podcasts-by-brainlid
- #ecto-query
- #elixir-ls
- #phoenix_html
- #iex
- #blog-post
- #graphql
- #genstage
- #ai
- #websockets
- #supervisor
- #advent-of-code
- #elixirconf-us
- #distillery
- #processes
- #forms
- #api
- #metaprogramming
- #security
- #performance
Our Sponsors
Build Elixir applications with speed and confidence.
Supporting innovation across the BEAM ecosystem.
We build reliable cloud platforms for business-critical systems.
Catch errors, track performance, monitor hosts and more.
Error tracking for Elixir devs who love to ship. Start your free account.
Develop your skills with books, videos, and courses.
Producing high quality Elixir screencasts since 2017.
Enabling companies to succeed by building software people love.
The team behind Membrane, Popcorn, LiveDebugger. Available for hire.
Courses that'll move you from confusion to "Aha, now I get it!"