Why Unit Test?

As a Microsoft Certified Trainer over the past 2 years, I’ve had the opportunity of teaching hundreds of professionals how to properly engage in unit testing.  Amazingly enough, even though unit testing has been around quite a while, it is still a concept that is very infantile in the corporate arena.  Most developers who have adopted unit testing are those who work for start-ups or the few companies who’ve started to embrace agile methodologies executing test-driven development (TDD).

Unfortunately, most companies are hesitant or event resistant to incorporating unit testing into their development process because they fear longer development cycles or less backlog churning over iteration.  These companies feel that unit testing does very little for the bottom line and, thus, needlessly increases costs for development.  Many studies have shown, however, that while unit testing does increase development time, the exact opposite is true about development costs and business value – long-term development costs decrease and business value increases both, short-term and long-term.  While I’m sure there are many more points to argue – and please do so in the comments – I’ll go ahead and discuss costs and value below in regards to unit testing.

Development Time

First, a question…What’s more important, quantity of work or quality of work?  With all love to my fellow developers out there, I mean no disrespect as I too am a developer, but let’s be honest… Developers for the most part are lazy.  As an architect, I’ve worked with many great teams and many not-so-great teams.  The difference wasn’t skill – after some experience, many developers are equal in the level of coding.  The difference is laziness or, lack of responsibility for their code.  The not-so-good developers are quick to simply satisfy a business requirement, but have not taken the time to test every permutation and therefore leave opportunities for the code the break.  The lazy developer’s mindset is, “it’s QA’s job to test.”  This is why TDD has been so eagerly adopted by start-ups – they don’t have the capital to invest in Quality Assurance Analysts (the developers have to test their own code).  In other words, the developers don’t take responsibility for their own code; they perform the minimum to get the application running and figure if there are any bugs, QA will squash them out.

While some companies have not yet incorporated unit testing, they have begun implementing policies such as gated check-ins.  In this case, if a developer’s code breaks the build (performed by an automated build controller), then their code is rejected in the source code repository.  The code must compile successfully in order to be an acceptable check-in.  Gated check-ins is a easier first step, but still requires a build controller to automate the build process and, even still, this is only a build test.  This type of test isn’t even considered an integration test, much less a unit test, because no functionality is actually being tested.  On the other hand, many build controllers can automatically execute unit tests and reject check-ins if those tests also fails.  By doing so, reduces the need for additional QA resources because if the unit test fails, there would be less time required to conduct integration testing.

So, going back to the original question, it’s unfortunate that most companies (and boards) are feature-driven.  They are more concerned with beating the competition with a plethora of new features and not necessarily delivering a quality product.  Their idea is that fixes will be performed through maintenance iterations.  What you have is a company with leadership who understands very little about software development – the fact that refactoring code sometimes costs more than initial implementation.  But, more about costs and business value in a minute.  This also breaks #5 of the Joel Test (a quasi-standard to software development being adopted by a growing audience), “Do you fix bugs before writing new code?” One more brief thing about costs, the costs employ large quantities of Quality Assurance Analysts, thus, again, raising their HR costs.  You want to see an optimized company?  Look at the ratio of QA to Developers.  In my experience, an optimized company should need no more than 1 QA to 4 developers.  Many start-ups, in fact, have 1 QA to 10-12 Developers.  This is, again, not simply because they’re forced to due to limited funds, but because their developers have an authentic desire to produce the best product.  If the ratio is anything less than 1:4, then that tells me that either the developers are not very well trained, or the QA Analysts are not well trained and therefore more than one QA is needed to perform an adequate job.  Either way, from a leadership perspective, some changes should be made – better developers or better QA.

Business Value

So, once again, returning to our original question, I would argue that quality of work increases business value, both in the short-term and long-term.  Quality improves short-term value because it delivers immediate gratification for the customer in purchasing upgrades which, in turn, drives sales (new or repeat) for the business.  If the customer knows and trusts the quality of a product, they will see the value of renewals and upgrades.  This is true with anything – not just software.  Specifically speaking of software though, if the customer’s trust is lost with a single release, this could potentially impact sales of future releases.  Need an example?  Just look at Windows Vista.  Quality of development – not merely quantity of features – directly impacts the bottom line when it comes to sales.  I would argue that if you continue to deliver a quality product, people will continue to stick with it no matter how slow the feature releases seem to be (you can even sometimes increase price, and people will still follow).  Another example, perhaps?  Look at Apple.  Their products are not business-class, they are slow to release new features, and their costs are sometimes 200%.  But, their developer’s and design guide is huge and must be strictly adhered to, or they don’t allow the release – even within their marketplace by other developers (e.g. iTunes).  It’s unfortunate, but Apple users have greater faith in their operating system than that of Microsoft users, especially the consumer side.

If quantity drives business value in the repeatable short-term by repetitive sales, how does quantity affect value in the long-term?  By decreasing overall costs to deliver the finished (e.g. solid, stable, etc.) product.  There are countless studies available that examine the costs of unit testing.  I read a study performed by Microsoft in the late 90’s (and there were other studies that concluded similar results) that unit testing may increase overall development costs by 60%. However, development costs were increased by as much as 40% for each bug found in production.  Read that again.  Development costs increased once by 60%, or repeatedly increased in multiples of 40% for each bug released into production.  While every developer knows this to be true, this is the fact that quantity, feature-driven leadership doesn’t get. The don’t understand the effort that is required to report the bug, validate the bug, fix the bug (often requiring code refactoring), then test the bug again, report any necessary changes to configuration/release management, configure the release, package the release, and deploy the release.  And, unless the company has implemented some type of automatic update mechanism (like Windows Update), users are continually plagued by the bug until they call Technical Support.  Then, costs are increased due to the increase in production support teams to assist users.  Not to mention the fact that, by this time, you have frustrated your user base, perhaps, even alienated some.  If this continues, consumers look to other offerings.  Two companies that come to mind who constantly do this are Intuit and Adobe.  While I also paid the extra $10 for the media just to have a physical copy, most of the time my computer couldn’t read the DVDs.  (Albeit, they’ve now reverted to a subscription basis.)  Therefore, I’d have to wait on the phone for, literally, hours to speak to someone in a call center who couldn’t speak or understand English just to ask for a replacement disc.  They couldn’t understand that I didn’t want (or need) a new license key, just the disc.  This was always an accepted reality because both companies had pretty good corners of their respected markets.  (Of course, Adobe’s products are great, but getting an installable hard copy was a pain.)

Furthermore, notice I answered finished product.  To executives and boards who look at quantity of features rather than quality of product, the product is “never finished” – they always want more.  However, developers who embrace code quality have agreed to a certain degree of completion, formally known as a “Definition of Done.”  This Definition of Done is not exclusive to Agile teams even though its utilized heavily in the methodology.  Regardless if a development team is conducting 2-4 week sprints or Waterfall cycles lasting 6 months to a year, responsible development teams agree to a level of completeness and don’t consider the product “finished” until this Definition of Done has been satisfied – no matter the number of features implemented.  For responsible teams, the finished, solid, stable product may only be the case for this two-week cycle, but it is thoroughly tested and can be confidently released.

Conclusion

It is advantageous for companies to explore the inclusion of unit testing within their development processes. Unit testing positively affects business value by delivering a quality solution that increases sales and customer retention while minimizing overall costs of development and production support.  If companies truly cared about their bottom lines, then why wouldn’t they want to reduce the costs to develop and support a product while increasing adoption and retention.  Like seeing a forest and desiring to cut down all the trees for lumber, but not realizing the time spent and equally wasted on cutting down the rotten trees, companies often times are focused on delivering faulty features that many times have little value to the customer and, therefore, produce little return on investment.  To put a different spin on an old adage, “Don’t miss the [good] trees for the forest.”  Or, in other words, companies should not overlook the value of a quality product for the quantity of its features.

NOTE: Because I’m always asked by my students how to write unit tests, I’m going to write a series on how to write various Visual Studio tests – unit tests, coded UI tests and load tests.  I will be writing these tests for MVC, JavaScript and WPF.  Additionally, I will be demonstrating how to perform dependency injection (DI) and inversion of control (IoC) within both, the MVC and WPF frameworks.  Finally, I will show how to mock data in order to properly test business logic as a unit without requiring a data repository.