tmbb

tmbb

Darwin - Mutation Testing for Elixr

I’ve been working on a mutation testing library for Elixir for a while now. It’s named Darwin, because it generates mutants in your code, which are then killed by your test suite (the analogy to evolution by natural selection should be pretty obvious at this point). The docs aren’t yet as organized as they should be, and test reporting is still very incomplete, so I haven’t released it on Hex yet. But the functionality is mostly done, and I thiknk it’s time I created a new thread in the Libraries category.

You can find the code on GitHub, along with an example on how to use Darwin on a real Elixir module.

Why another mutation testing library?

We already have Exavier by @dnlserrano, so why another one? First, both libraries are still in a very early development stage, so it’s not clear at this point that one of them is clearly better and the other one should be abandoned. Second, both libraries follow very different design philosophies, so they can’t exactly be merged or something like that.

I plan on writing a series of posts comparing Exavier to Darwin and defending some of Darwin’s design decisions.

Can I use Darwin in my real project right now?

Probably yes. Code is only mutated in development, so your production code should be unaffected. On the other hand, Darwin has a number of problems:

  1. Test reporting is not great
  2. Running all your mutation tests might take a while
  3. Darwin uses some nasty private ExUnit APIs (Exavier does the same, by the way; ExUnit is not really suitable for use as a library)

Can I use Darwin for Mutation Testing of Erlang Code?

Currenctly HTML reporting is not supported for .erl files (only .ex) files. This can be changed though. Darwin mutates the code at the level of the fully expanded Erlang AST, so it should be an excellent choice to mutate Erlang libraries. The main drawback in this use case is that you have to install Elixir and use ExUnit in order to test mutations in your own Erlang project.

It’s likely that Darwin will be easy to integrate with pure erlang test frameworks, as long as someone writes the necessary glue code. Sometimes I think of rewriting Darwin in Erlang, but Elixir’s conveniences make it much easier to write in Elixir than in Erlang, so I don’t think I’ll go that way until Darwin is stable enough.

Showing Posts 1 to 6

OvermindDL1

OvermindDL1

Ooo it’s released, awesome!

tmbb

tmbb OP

It’s not released. You can’t find it on hex yet. It’s just stable enough that I’ve decided to create a new thread in the forum

tmbb

tmbb OP

Documentation is a bit better. Most mutators are still undocumented, but that’s something I’ll solve in time. I’m also writing a series of articles about Darwin’s internals for future contributors (and because Darwin’s ideas can be applied to other kinds of projects that manipulate Erlang’s source, or the source of any functional language, for that matter)

Darwin can now do the following:

  • Mutate arithmetic and comparison operators
  • Mutate some literals (not all literals yet)
  • Traverse Erlang AST and apply mutations recursively
  • Mutate code in guard by recompiling function clauses into a linear sequence of matches that doesn’t use any guards; it effectively moves guards into “normal code”, where you don’t have any restrictions on what kinds of expressions are allowed. It’s interesting because guards and pattern matches are actually not very composable… You have to turn them into (very inefficient) normal code in order to make them easier to work with. The inefficiency is probably not a dealbreaker because Darwin is only used as a test tool and not in production.

There are still some useful mutators missing. However, if you think you’re missing any mutator, you can just implement your own (mutators are behaviours, and you can add your own modules that follow the behaviour)

axelson

axelson

Scenic Core Team

Cool, I’ve been following along with the development and mutation testing seems like a neat idea. So is Darwin ready to try out on actual projects now?

tmbb

tmbb OP

Yes, you can try it right now, but the test reporters are still very primitive. They merely list the mutations and whether they have survived or not, and don’t gather any statistics yet

tmbb

tmbb OP

Test reporting is now better: it produces a pretty HTML file with mutation listed by line.

It can now perform more mutations too. It can now perform mutations inside pattern matched, which it couldn’t previously. It does so by rewriting pattern matching expressions as a chain of function calls, which is less efficient but I don’t think it matters much when it’s only used for testing.

— All posts loaded —

Where Next? Top

Trending in Announcing Top

GenericJam
Edit: 2026 May 15 - This post is archived. Mob is alive!! Main docs: mob v0.7.11 — Documentation A bit of explanation for the slightly c...
261 13183 112
New
achempion
Hi, I would like to tell about my initiative to further maintain and develop Waffle project which is the fork of Arc library. The progre...
New
mudasobwa
I fully migrated to my own harness from Anthropic/Gemini and I think it’s time to share it. Welcome DSH, the DeepSeek Harness, fully writ...
New
kip
Localize is the next generation localisation library for Elixir. Think of it as ex_cldr version 3.0. The first version will be released ...
New
jimsynz
Beam Bots (or just BB for short) is a framework for building fault-tolerant robotics applications in Elixir using familiar OTP patterns. ...
New
mcass19
ExRatatui lets you cook up rich terminal UIs in Elixir, powered by Rust’s ratatui via Rustler NIFs. Build interactive terminal applicatio...
New
charlenopires
Hi everyone :waving_hand: I’ve been working on PhiaUI, an open-source UI component library for Phoenix LiveView, and I wanted to share i...
New

Other Trending Topics Top

OndrejValenta
(I just needed to vent somewhere and LinkedIn is full of hope, or hype, I’m not sure which exactly) AI dream has many faces, but general...
New
bartblast
If you haven’t tried Hologram yet, what’s stopped you? Here’s why I’m asking. Building a framework turns out to be two jobs, the framewo...
New
mat-hek
Popcorn, the library that runs Elixir code in the browser, is now stable :popcorn: This is possible because it now builds on a very stab...
New
AstonJ
With advancements in AI gathering pace and set to increasingly accelerate, perhaps it’s worth having an annual thread on the topic - feel...
New
sntran
In my third year of college, around 2010, I found Erlang. A few years later, I found Elixir. I liked the ideas at once: many small proces...
New
gtcode
I’ve been tinkering with Lynx (by José) via hold_ledger. I used AI to write the code, a separate specification, the tests, and the Lean p...
New

We're in Beta

About us Mission Statement

Options

Thread Display Mode




Thread Preview

Skip Thread Previews