I really appreciate your views on tests and agile and pretty much the software culture as it is right now, but I absolutely disagree with you in almost point you make.
This is probably a great example of how there is no absolute right way to build a house, but you can disagree in the details.
It sounds like you are suggesting that unit tests are worthless, and should always be disregarded in favor of people using the software. I think this is wrong, and I will make my case further down. I have written software with and without unit testing, and I know the pain tight unit testing can inflict on the overall developing process, so I do not agree with dogmatic TDD approach. But I also don’t believe that is how people practice TDD.
Furthermore, you mentioned Uncle Bob - he has been a person I used to look up to, because what he said made sense to me, and following some of the stuff he preached made me feel better as a developer and still - I agree, he is a dogmatic “do never change my gospel” kind of person, and some of the more recent things he said are even more disgusting. But you can not disprove things by quoting people who might have done wrong or not, that is an “argumentum ad hominem”, and known as a fallacious argument.
You also quote passages of studies that support your argument, but disregard other aspects of the argument that are way outside of the study, which is another fallacious way of arguing your point you keep using.
I realize this might come off as a personal attack, but it’s not. I just think you are wrong, and you will probably think I am wrong. That’s healthy, and that’s how it should be! You may correct me in any of the points I want to make, if I misrepresent your opinion.
So now to my actual points regarding Code Inspection VS Unit Tests.
1. Unit tests constrict development, and make changes harder
I enjoy writing unit tests, because I like watching red turn to green as a feedback for my efforts. I agree that unit tests can distract you from the actual goal, and in a test suit that was not done with unit tests in mind, the test setup can be far too complicated, just to prove that - indeed -
plus(1, 3) returns 4 (if you have unit tests that basically just test usage of basic implementations like this, or implementations that have been tested in a library, delete them.)
Maybe you could say that I am an advocate of deleting tests after writing them. If you want to make sure 1 + 3 makes 4 in your setup… Sure go ahead! But don’t expect anyone (including yourself) to maintain that test, at some point someone will raise the question whether or not you should even have such a function.
I am doing different approaches to software development, I work with 0% test coverage, I work with close to 100% test coverage, I work with code that had tests written after the code, and I work with test written before. I almost certainly do not have your level of experience (sorry for making deductions based on your avatar), but I can certainly say that I am thankful for everyone who bothered to write tests, even if the tests are crap whenever I have to dive into unknown code.
Because their tests describe their thought process. I make a change, suddenly 50 tests are red… Oh oh, I did an oopsie? No, they just wrote all their tests assuming that the arguments will always be exactly X. Ok. Bad assumption, and I have to rewrite the tests anyways to fit my change. And suddenly there is only 1 red test, and I realize that I would break an existing feature with what I did.
I lost some time, but I gained a lot of insight of how the person who wrote this code thought about the process. It can be super frustrating - WHY WOULD YOU - mea culpa - we are all very guilty of writing crappy code from time to time.
However, working my way down from the top level function, to the more nitty-gritty ones, and having tests supporting my assumptions, helps me think about the over-all design, and it helps me rethink! This is my opinion, and it certainly works different for different people, but whenever I break down a function into its components, and I realize that the setup for even one of those unit tests is extremely difficult, then I might have gone into the wrong direction…
This is imo what doctests in Elixir promote. If you can not test it in a doctest, do you really want to do it like this? Should this not be a top-level function? I can not count the number of times I realized that a function should be top level, and thus tested in a more integration way, than unit way, … I can’t count it.
I said that I would advocate for deleting unit tests after writing them, and I think that is a very valid approach. If you are like me, you would want some very small scale verification on parts of your code working. If you are like me, the process of writing
test "it does exactly X if given Y"
helps you think about what your function should and should NOT be doing, and improve code quality. But it might not be beneficial to make code future proof, so by all means… Get rid of development tests! Tests are lines of code that have to be maintained, less code often means less maintenance - yes!
But by no means I think unit tests are a waste of time or could be replaced by pure integration testing, be it by humans or a machine. What you are suggesting is that libraries should be tested by their users, contributors be damned.
2. Code Inspection makes Unit Tests obsolete
*Dislcaimer: I assume we are talking about code inspection as defined by Wikipedia If I am wrong here, I am looking forward to reading what your definition is.
Not only does this disregard my point about how unit tests can help you thinking about code, it also suggests that everyone can afford code inspection.
What if you are a startup, and you think your code could be improved by the masses! You make a library public, and boom, there is your code inspection, right!!!
No, that is not how it works… People will hesitate to contribute to libraries they don’t fully understand, tests will give them security, etc. But I understand you are talking more about specific projects, where everyone is working for a single goal to achieve! Like, that company will never have people leaving, people joining!
Think about a developer who has to do a simple task, in a big project. Would it make them feel more confident about their code if you told them “we will know if it worked or not once we have the Code Inspection results”, or if they could run a test suit that either tells them “all green” or “10 red”?
If you think just doing things in a project that is completely foreign to you, without a tight test frame is completely fine, then you are a hypocrite. I have never met a junior developer, or a senior, who was fine with just changing stuff to what they thought was right, without getting feedback from anyone - or in the case I am trying to make - something.
3. Developers save time by not writing unit tests
This is a very, very controversial point. I have lost hours and hours just fixing tests that did not meet the initial expectations anymore, but I - as in, me personally, people are different - prefer delving into how things USED to be and potentially find bugs before they happen, over months of calm sea and then facing the ultimate storm.
Maybe you are not involved with these kind of things, but web developers have to face the ugly truth of no version can last forever. Upgrading an Ubuntu server to the latest version might feel great, but how long until the app has to be updated? Are you running an Elixir 1.6 app? That is so outdated… Have fun updating to 1.10 (disclaimer, I found Elixir apps to be super easy to update, since I prefer to pull in the least amount of dependencies, and Elixir tends to give very descriptive warnings, but boy… there are some tricky bastards to tackle already!)
A good test suit can give great developers a great peace of mind - (at least I did not break anything.)
4. TDD takes the fun out of development
Now this one is entirely me, putting words into @StefanHoutzager 's mouth. It always sounds like you are saying writing tests first takes away from the developer experience and freedom, basically just testing it manually and throwing the result at users is test enough. Sorry if I misinterpreted you, feel free to correct me.
I confess, I do not always write tests first. I confess I think testing certain template responses in Phoenix (or whatever framework) is a waste of time. I confess I think Uncle Bob is full of ***. I confess I think some of the projects I know like the back of my hands should quit bothering me with implementation details, badly written tests and just let me deploy what I (at that certain time think) is the fix for a bug!
But I generally enjoy the red -> green cycle of tests. I enjoy writing stuff that I know others will look at, my test descriptions, and I enjoy their feedback. I am guilty of writing tests like “it works with X”. I am guilty of overtesting. I am guilty of not testing enough.
I usually am more thankful for my over-testing past self though. At least it forces me to understand why a test was failing!
5. What are you proposing then…?
My approach would raise eyebrows in every community. Me, my next year me, would probably scold me for what I am doing right now.
I write tests… I do your approach from time to time (without the million dollar company you seem to have behind you, or whatever it is that lends you your ego) and just go for it.
I sometimes prefer writing tests after I figured out the top level API.
I sometimes hate myself for the unit tests I wrote.
I sometimes love myself for the unit tests I wrote.
I sometimes omit top level tests.
I write code without any tests!
I will not merge any code without tests into this project!
This is how programmers work, we are all human, we all prefer different approaches. But I think you are wrong @StefanHoutzager