lol I see I managed to snipe both of you with this throwaway line.
Hobbes’s development is absolutely driven by testing. In fact, it’s test-driven to a frankly extreme degree. But it is not “Test Driven Development” in the popular sense. The first line for TDD on wikipedia literally explicitly mentions “unit tests” so I feel pretty justified that that’s the popular sense.
And I have written ExUnit tests as a means to develop features sometimes, but they are not unit tests. They are serving as stand-ins for a script, with the sole reason being that mix test is easier to type than mix run priv/scripts/whatever.exs. There is no resemblance to actual unit testing.
I did not have such an extreme opinion before working on this project, but unit tests are completely and utterly worthless. They are best used as a manual method of snapshot testing to catch regressions, but one would be better off doing actual snapshot testing there.
Once you are working on a program which is complex enough that you cannot exhaustively validate its correctness in your head, you need to start fuzzing. Every database-shaped codebase I have read (which is, at this point, many) has had fuzzers. I don’t know why people don’t talk about this more, but I guess it’s just something that you learn pretty quickly while working on such code.
Unit tests do not find bugs. They suck.
