I am quite guilty of allowing myself to fall into the mentality that there is a solution to software problems that can handle all situations and solve every problem. Unit testing is certainly one of those things that I felt makes all the difference in the world and is a major requirement. When we started development here, I required full unit testing, used code coverage tools to make sure. However, over time I became quite lax on making sure everyone wrote their tests. Part of it was laziness (writing Unit Test is perhaps the worst, most boring aspect of any development), part of it was it was feeling like a bit of a waste, and lastly, it wasn’t catching errors. That’s right, if our Unit Tests were passing, then why in the hell are we having bugs reported?
Wil Shipley summed it up quite nicely here. He is quite opposed to Unit testing, and some of his comments really hit home for me.
There’s only three major flaws with this: (1) Essentially, to write a program that fully tests your program, you need to encapsulate all of your functionality in the test program, which means you’re writing ALL THE CODE you wrote for the original program plus some more test stuff, (2) YOUR PROGRAM IS NOT GOING TO BE USED BY OTHER PROGRAMS, it’s going to be used by people, and (3) It’s actually provably impossible to test your program with every conceivable type of input programmatically, but if you test by hand you can change the input in ways that you, the programmer, know might be prone to error.
These sum up what I’ve been starting to feel, but he drives his point home with
Unit testing doesn’t give you information on how common a bug is, which is the single-most important piece of information about a bug, followed very closely by its severity.
Now, in his first quote he mentions the fact that “your program is not going to be used by other programs”. This, of course, is not always true.
Bill Bumgarner counters with a good argument of when this case occurs and how unit testing can solve a problem.
So, it seems the answer is: Writing an API? Unit Test. Writing an App? Don’t waste your time.
EDIT
bbum made a comment about this post (although, when he attempted to do so, comments were broken), so I’ll update the post here.
[Ed:] Unfortunately, the mojomonkeycoding.com weblog’s comment system is busted. The author of that post missed the point. If he had been paying attention, he would realize that unit testing is critical in a well designed app in that you will have a model/control layer that is a great UT target.
I am a bit confused as to which point I missed. From his post he speaks of the benefits unit testing Core Data to allow applications such as Delicious Library to trust that they can read/write data successfully. That’s to be expected, as the author of a core library unit testing make perfect sense.
And that is the answer. A software developer like Wil — a totally user focused, user experience driven, engineer — may not see the value of using unit testing directly in their development efforts. However, that Delicious Library could use something like Core Data and does use all of the other technologies in Tiger would not be possible without the huge investment in Unit Testing by the various teams at Apple.
Right, so because you so diligently tested the core API that it is using, he is able to focus on the user experience. Therefore, he doesn’t need to write those unit tests, instead relying on functional tests and focus testing.
Perhaps, my conclusion was a bit overly generalized:
So, it seems the answer is: Writing an API? Unit Test. Writing an App? Don’t waste your time.
I can see how this statement may seem obtuse, as it was a bit tongue-in-cheek, and “App” should have made it clear that I’m talking about the End User Application (the application may contain its own APIs). However, I still feel that if you are writing underlying code that client’s use, Unit Testing make complete sense. But when writing a user facing application, getting it in front of user’s (yourself included) is better use of your time.
Lastly, all that being said, I am starting to swing back to the other side, thinking more about TDD, etc. As the tools get better to make this stuff easier it makes it more inviting.
