ancatrusca

ancatrusca

New episode with Tobias Pfeiffer (PragTob) covering Benchee’s design and the systematic mistakes that make most benchmarks misleading.

Practically useful things from the episode:

Inputs as first-class. Test your benchmarks against multiple input sizes and shapes -empty list, small, medium, large -before you trust the result. Tobias built input support into Benchee after deploying an “optimized” query that performed worse in production because the benchmark had only tested one data shape.

Latency vs. throughput. These are different measurements. Latency is how long a single operation takes. Throughput is how many operations complete per unit time. For a warm web application, throughput usually matters more. Most benchmarks measure latency and draw throughput conclusions.

Reductions. A more stable measurement than wall time for comparing implementations. Deterministic within a given OTP/Elixir version, machine-independent. Benchee reports them alongside time and memory. If reduction count is stable but wall time varies, the noise is environmental.

No macros. Benchee is functions all the way down. This was deliberate -functions can be wrapped in macros by anyone who wants them, but macros can’t be easily turned into composable functions. The design choice is what made Benchee extensible enough to have CSV, HTML, markdown, and LiveView formatters written by the community.

Where Next? Top

Trending in Podcasts Top

ancatrusca
The setup Hernán Rivas Acosta (Erlang Solutions) built Spawndemonium, a top-down multiplayer shooter shown at Code BEAM Europe. The simu...
New
PJUllrich
The eMudhra root certificates were compromised, and we cover how to check whether your Elixir app is affected. Elixir 1.20.5 fixes three ...
New
ancatrusca
What Bondy is A declarative, statically typed language for the BEAM that combines Datalog and Constraint Handling Rules, created by Alej...
New
ancatrusca
Mike Hostetler joined Allen and Francesco to talk about Jido 2.0, what he rebuilt from 1.0, and why the framework still keeps the LLM out...
New
PJUllrich
Hex 2.6.0 adds Workflow Identify aka. Trusted Publishing, staged publishing, scoped API keys, and an emergency approval mode to reduce pa...
New

Other Trending Topics Top

GenericJam
Edit: 2026 May 15 - This post is archived. Mob is alive!! Main docs: mob v0.7.11 — Documentation A bit of explanation for the slightly c...
261 13183 112
New
OndrejValenta
(I just needed to vent somewhere and LinkedIn is full of hope, or hype, I’m not sure which exactly) AI dream has many faces, but general...
New
kip
Localize is the next generation localisation library for Elixir. Think of it as ex_cldr version 3.0. The first version will be released ...
New
jimsynz
Beam Bots (or just BB for short) is a framework for building fault-tolerant robotics applications in Elixir using familiar OTP patterns. ...
New
mcass19
ExRatatui lets you cook up rich terminal UIs in Elixir, powered by Rust’s ratatui via Rustler NIFs. Build interactive terminal applicatio...
New
charlenopires
Hi everyone :waving_hand: I’ve been working on PhiaUI, an open-source UI component library for Phoenix LiveView, and I wanted to share i...
New

We're in Beta

About us Mission Statement

Options

Thread Display Mode




Thread Preview

Skip Thread Previews