Structured Generation and Logits Processing with Elixir - Chris Beck | Code BEAM Europe

[CodeSync]

Structured Generation and Logits Processing with Elixir - Chris Beck | Code BEAM Europe 2025

Structured Generation and Logits Processing with Elixir - Chris Beck | Code BEAM Europe 2025

This talk was recorded at Code BEAM Europe in November 2025. If you’re curious about our upcoming event, check https://codebebeameurope.com

When we call an LLM from an application, the result can be a gamble: will it follow the schema, include every required field, or even produce valid JSON? Today we usually validate after generation. If it fails, trigger another generation round that may still break.

There is an alternative: By reaching into the raw, unnormalised token scores (logits) during decoding, we can prune every token that violates a grammar we supply. The model literally cannot create invalid output; no retries, no patch-ups.

This logits-level gating was first published in 2016, yet it remains largely unknown outside a handful of research papers. I’ll surface the idea, show why it slipped under industry radar, and—most importantly—demonstrate how easily you can apply it today with Elixir’s Bumblebee, Nx, and the BEAM’s supervision strengths.

We will see how the technique makes small but critical tasks reliable —emitting strict JSON configs, creating test data, or generating Erlang term syntax—where unconstrained decoding would crumble.
You’ll leave knowing how to wire grammar constraints into Bumblebee, eliminate brittle post-processing, and ship more reliable, safer Elixir systems.

Comments welcome! View the code-sync tag for more Code Sync talks!