Mutation testing - Mutating BEAM bytecode

After reading @dnlserrano’s post about his new exavier library (which actually has an end-to-end system which you can run now, and a much cooler name IMO), I’ve decided to work a little more on Darwin. We must remember that Darwin is commited to using mutant schemata (as explained above). This means that, by design, Darwin has the following problems:

  1. I might mutate code deep inside some macro in a way that’s very opaque to the user. I’m not even sure that it will be easy to detect erlang code expanded by macros as compared to code compiled from “normal elixir” code.
  2. I can’t replace operators in guards (for very good reasons, erland doesn’t allow arbitrary functions inside guards). Guards often contain comparison operators and are critical to the control flow of the program.

Point 1 seems unsolvable, except by clever heuristics.

Point 2, on the other hand, seems solvable. Most (all) code with guards can be compiled to code without guards and with nested case statements replacing the guards. Because guards are all nice and functional, they can be manipulated in a safe way (probably killing performance, though). This seems tractable with only a moderate amount of effort.