What @hauleth wrote is correct, but note that I am using the term a bit more broadly. Fuzzing, property-based testing (PBT), and deterministic simulation testing (DST) are all approximately the same thing. I’m using “fuzzer” to describe all three of them because the other two terms are acronyms and I don’t like acronyms. This is also consistent with Tigerbeetle’s usage of the term, e.g. in this article.
Simulation tests are essentially the same thing as property tests except that for distributed systems it’s very hard to make those tests deterministic. DST is a set of techniques for making property tests deterministic by simulating the things that would be nondeterminstic. In the case of Hobbes, that consists mostly of removing parallelism, simulating the disk, and making sure all calls to random() are properly seeded. (And also avoiding iteration over maps, which has been the source of 100% of the determinism bugs I’ve had in practice lmao)
There are also fuzzers that are domain-specific. I originally started writing them while working on the KV data structures because I realized it was literally impossible to write useful unit tests for them. XKS (the storage engine) currently has 5 different fuzzers, and will probably need more.
I’m working on a new fuzz/simulation framework named Trinity to replace Construct, as the latter was not designed with performance in mind (it really was not “designed” at all). I’m hoping I can get a good performance boost as I need to run much longer simulations to validate Hobbes for production use.
