gtcode

gtcode

I’ve been tinkering with Lynx (by José) via hold_ledger. I used AI to write the code, a separate specification, the tests, and the Lean proofs.

LYNX

Write proofs about Erlang/Elixir programs using Lean.

This is done by writing laws, in Erlang/Elixir, with a Lean proof that guarantees your Erlang/Elixir code obeys those laws. The laws are automatically translated to Lean and the proofs are checked against a model of the Erlang runtime written in Lean.

Lynx implements most Erlang/Elixir language constructs, although several built-in functions (BIFs) may be missing. The focus is on verifying sequential programs although concurrency constructs (spawn, receive, send) are also modelled. Feel free to open an issue for missing functionality.

See the documentation for the Lynx module to get started.

Naturally, everything tests green: Dialyzer, Credo, tests, and all 29 formal law groups. I’m sure most people here have access to frontier LLMs and are aware that well-tested code is broadly available. But how much confidence should I take from this? AI wrote the spec, too. And, of course, no bugs in the original implementation were uncovered during all of this.

Granted, there’s something circular about devising an example like this and writing proofs for it. Perhaps a better experiment would have used an existing codebase. I believe that’s a primary intended use case for Lynx, after all. It would be cool to see agents exploring existing libraries, suggesting laws, attempting proofs, and investigating counterexamples.

The more I’ve thought about it, the more Lynx’s current approach makes sense. It works with actual compiled Erlang/Elixir code, and agents can handle ginormous Lean proofs.

The recent Anthropic/Theorem work generated massive Rocq proofs across over 100 libraries. Yes, it’s a different approach. What becomes practical when agents can do the proof-writing?

hold_ledger also made the verification boundary quite obvious. It proved properties of the pure state machine, not the GenServer, journal, external processor, or their interactions. I suspect the human’s job will be to review the laws, assumptions, and boundaries of what’s been proven.

Anyway, having worked on this codebase today, and from reading chatter online, I anticipate that formal verification will become first class in coding harnesses.

I was able to prove properties for a state machine. Pretty cool. I’m excited to see where Lynx might go in terms of OTP / process interactions / delays / retries / crashes / persistence / effects / etc.

I’m curious whether that’s roughly how others see Lynx developing – more as a tool for agents to investigate and establish properties of real Erlang/Elixir programs than as a traditional verification environment that developers operate by hand?

And if AI can produce the implementation, specification, and proof, what would you actually want a human to review before trusting the result?

Showing Posts 1 to 6

caleb-bb

caleb-bb

Well, this is just what the doctor ordered! I guess I have an excuse to learn Lean now. :joy_cat:

arcanemachine

arcanemachine

Nice, how did I miss this? I was just thinking about dabbling in this space.

Thank you José (and to @gtcode for pointing this out)!

gtcode

gtcode OP

Quick update: I’ve been noodling with a review workbench:

Review Web README

The goal is to make it easy to reason about Lean 4 verification without needing expertise in proof engineering, all in the context of the repo’s business payment example.

I’ve been building this for me, not for general consumption. No Dockerfile, etc, for the main project’s bootstrap in the repo’s root, but you can easily adjust if you’d like to tinker. Also, the Review Web Phoenix app requires Elixir >= 1.20.4.

Having said that, any feedback is appreciated.

mudasobwa

mudasobwa

Creator of Cure

To me it sounds like “to judge economics without having any idea what the multiplication table is.”

Also, I’d say it”s worth mention that Lean proof does technically remain a soundness proof if and only if the target code in Lean.

gtcode

gtcode OP

Yes, fair. It was a poor choice of words. The goal is to expedite understanding, not skip it. That is a vision but is far off, and I am unaware of any substitute for reading and using code to understand it.

I meant to say something about not needing to be an expert proof writer to review claims vs. behavior. That’s an aspiration, to be sure.

I trust José’s Lynx implementation, and it’s reasonable to expect convergence towards trustworthiness. Of course, that’s not the same as a formal guarantee, so your point is well taken.

The objective is to help humans review what’s being proved, not certify the entire verification toolchain.

gtcode

gtcode OP

Lynx was updated this week. The biggest change that I observed pertains to the manner by which and/or expressions are translated into Lean. Simplification rules have been added. Those changes affected proofs in my hold_ledger trainer. The biggest new capability, generally, is FV across all modeled process interleavings, so we can prove properties across scheduling choices.

Given the manner in which Lynx is progressing, I’m excited to see how far José goes before committing heavily to any additional tooling. Having said that, I am knee-deep tinkering with the ecosystem and seeing what makes sense for a defined process that combines FV with OTP concurrency/resilience testing. I’m working on a private project establishing a workflow with existing tools, while considering Lynx to be a moving target. Goal is clear: Establish a development process where AI builds complex Elixir/OTP systems using proofs, more systematic testing, and transparent evidence for claims.

Moving on, Microsoft Research added a blog post for Proof Driven Development. The blog post says it’s like spec-kit but for FV rather than Spec Driven Development. It’s a limited research preview, but the blog post is excellent for mining ideas / inspiration.

I’d love to keep this thread going, because FV is, IMHO, the most consequential topic now and for the foreseeable, with Lynx at the center for Elixir.

— All posts loaded —

Where Next? Top

Trending in Discussions Top

AstonJ
The obligatory hello world thread! Who are you and where are you from? :stuck_out_tongue:
4663 61218 595
New
cblavier
Hey there, It’s been more than a year since we started using LiveView as our main UI library and building a whole library of UI componen...
515 12931 101
New
wmnnd
Hey folks, I wanted to share with you a project I’ve been working on for a while: It’s an Open Source (AGPLv3) newsletter tool. Think Ma...
New
raa-ziq
Hi everyone :waving_hand: I wanted to share something I’ve been building — NowBlind, a random chat app designed to help people discover ...
New
sntran
In my third year of college, around 2010, I found Erlang. A few years later, I found Elixir. I liked the ideas at once: many small proces...
New
gtcode
I’ve been tinkering with Lynx (by José) via hold_ledger. I used AI to write the code, a separate specification, the tests, and the Lean p...
New
Ganna
Hi everyone, I have 8 years of production experience with Erlang/OTP and I’m just starting to get to know Elixir. What would you learn f...
New

Other Trending Topics Top

dominicletz
Hey there, I haven’t seen a list of useful Elixir specific skills for the popular AI tools yet so I’m starting one here with my first. ...
New
mudasobwa
I fully migrated to my own harness from Anthropic/Gemini and I think it’s time to share it. Welcome DSH, the DeepSeek Harness, fully writ...
New
KristerV
Hey. Is there anyone here who creates agents in their apps? Not talking about using agents, but creating them. I’m finding it pretty diff...
New
OndrejValenta
(I just needed to vent somewhere and LinkedIn is full of hope, or hype, I’m not sure which exactly) AI dream has many faces, but general...
New
kip
Localize is the next generation localisation library for Elixir. Think of it as ex_cldr version 3.0. The first version will be released ...
New
jimsynz
Beam Bots (or just BB for short) is a framework for building fault-tolerant robotics applications in Elixir using familiar OTP patterns. ...
New

We're in Beta

About us Mission Statement

Options

Thread Display Mode




Thread Preview

Skip Thread Previews