ancatrusca
New episode with Tobias Pfeiffer (PragTob) covering Benchee’s design and the systematic mistakes that make most benchmarks misleading.
Practically useful things from the episode:
Inputs as first-class. Test your benchmarks against multiple input sizes and shapes -empty list, small, medium, large -before you trust the result. Tobias built input support into Benchee after deploying an “optimized” query that performed worse in production because the benchmark had only tested one data shape.
Latency vs. throughput. These are different measurements. Latency is how long a single operation takes. Throughput is how many operations complete per unit time. For a warm web application, throughput usually matters more. Most benchmarks measure latency and draw throughput conclusions.
Reductions. A more stable measurement than wall time for comparing implementations. Deterministic within a given OTP/Elixir version, machine-independent. Benchee reports them alongside time and memory. If reduction count is stable but wall time varies, the noise is environmental.
No macros. Benchee is functions all the way down. This was deliberate -functions can be wrapped in macros by anyone who wants them, but macros can’t be easily turned into composable functions. The design choice is what made Benchee extensible enough to have CSV, HTML, markdown, and LiveView formatters written by the community.
Trending in Podcasts
Other Trending Topics
Categories:
Sub Categories:
Forums
Popular Tags
- #ecto
- #liveview
- #troubleshooting
- #learning-elixir
- #library
- #deployment
- #erlang
- #testing
- #genserver
- #mix
- #absinthe
- #remote-other
- #otp
- #plug
- #how-to-question
- #macros
- #postgres
- #elixirconf
- #channels
- #exunit
- #code-sync
- #discussion
- #podcasts
- #javascript
- #onsite
- #dialyzer
- #docker
- #authentication
- #ai
- #elixirconf-us
- #umbrella
- #full-time-contract
- #podcasts-by-brainlid
- #blog-post
- #ecto-query
- #elixir-ls
- #phoenix_html
- #iex
- #graphql
- #genstage
- #websockets
- #supervisor
- #advent-of-code
- #distillery
- #security
- #elixirconf-eu
- #processes
- #api
- #forms
- #hex









