Importance of independent testing

In this age of TDD, people are getting an impression that TDD or other forms of developer-testing virtually guarantee absence of bugs. They think that 100% code coverage means that bugs have nowhere to hide.

The problem is that developer-tests are made with the same assumptions and logical errors as the code itself. If you don’t think about a particular scenario that would cause your code to malfunction, you are not going to write a test for it.

Of course, modules/classes/services can and should be unit and integration-tested. That provides guarantee against basic errors and allows conducting safe code-refactoring by verifying that existing functionality is not broken. Personally I’m a big fan of integration-testing and intend to blog on it later.

People should also keep in mind that despite wide-spread beliefs in extereme effectiveness of developer-testing, empirical studies show moderate rate of bugs detection though developer-tests. In Code Complete 2 Steve McConnel reports that “Individual testing steps (unit test, component test, and integration test) typically find less than 50 percent of the errors present each. The combination of testing steps often finds less than 60 percent of the errors present (Jones 1998).”[McConnell, 2004]

That compares with about 80% detection rate achieved by modeling and prototyping, and 75% detection rate achieved by formal code reviews. You might not agree with the conclusions but the numbers are accumulated over a large number of projects and chances are that they are at least somewhat reflective of the reality.

I truly believe that in addition to all developer testing (include unit-testing, reviews and prototyping), independent QA/Functional testing is indispensable in finding real bugs that would otherwise make it into the production.

First of all, a person or a team of testers are free from the assumptions that led to the bugs in the first place. Second, a fresh pair of eyes and an experience (and talent) in finding faults of the system are what is needed to uncover the dark corners of the software.

Third, I’m a firm believer that a division of labor leads to a higher productivity. Without going into long economic theories, it is known that people that specialize in a particular skill tend to be more productive in it. From this stems the overall increased productivity of the team and the company. A tester is a specialist, well…in testing. That’s what he knows to do best. Just as a developer, he accumulated a bag of techniques and tricks that help him solve testing puzzles. Because of this he can find more bugs, more boundary-conditions, more unexpected inputs than the developer.

For these reasons I believe that rigorous testing by an independent team of testers is absolutely necessary in addition to all forms of developer-testing.

Steve McConnell rules!

As person with over 15 years in software business I must admit that the biggest mistake I made is that I assumed that general programming and later management knowledge just comes to you with the experience.

Even if it is partially so, you’re inevitably exposed to a relatively small sample of problems and solutions. Furthermore, you’re bound to learn only from your own mistakes. In many cases people see those mistakes and the mistakes made by other people around them (including management) and assume that these mistakes are somehow unique to their project or their company.

But as the saying goes: “There is nothing new under the sun” (Ecclesiastes 1:9). Most of the mistakes that you or others around you made had already been made countless times before. It pays to read serious literature about avoiding such mistakes through use of best software development practices. Further more, this literatures should preferably be read before engaging is big development projects.

After reading quite a few books on general subjects of software development (besides technology and language-specific books) I can say that I found most valuable information in the books of Steve McConnell.

When I started reading “Code Complete 2” – it was love at first sight. The book made me completely rethink my approach to writing code. The book includes both low level advice such as recommended naming conventions and sizes of modules as well as high level concepts such as managing complexity though consistent levels of abstraction that is essential to intelligibility of systems. I believe that any serious software professional ought to read this book, preferably before he spends years on making and correcting his own mistakes.

The next book by McConnell that is more intended to team-leads/architects is “Rapid Development”. Although the book was published in 1996, it’s still very relevant today. Concepts and ideas in this book apply to today’s “agile” world as much as they did to “waterfall” world of past decades. It contains an invaluable list of practices and classic mistakes that every developer/manager encountered at some point. Same as with Code Complete 2, this book is essential reading for any professional software practitioner.

A more specialized book by McConnell is “Software Estimation”. The book contains exhaustive description of various estimation methodologies along with advice on applicability of each methodology within a given context. It also contains information on estimation of “iterative” (aka agile) projects. I believe that serious software estimation is impossible without basic knowledge of techniques described in this book.

A common feature of McConnell’s books is reliance on large studies of projects and code-bases. This is particularly valuable because it takes the information from realm of personal experience and opinion into realm of scientific and empirical knowledge that is so much missing in many books on software. I don’t want to start a flame war or invite angry feedback but books such as Uncle Bob’s “Clean Code” lack this kind of empirical proof and are mostly based on author’s opinion (with which I mostly agree, btw). I believe that software profession has evolved for long enough time that we can use information accumulated in countless successful and failed projects to better plan and execute new projects.

Lastly, I regret that very few colleges and universities include McConnell’s books as part of mandatory learning material. IMHO they contain infinitely more useful information to future software engineers than many abstract and outdated subjects in computer science.

Caffeine Dependency is Easier to Develop Than You Think [Caffeine]

This article confirms my personal observation:
"Does Coffee Work" for Seed Magazine, and comes to mostly the same conclusion we did: yes, if you don't drink it regularly, and go slow with it.

Caffeine Dependency is Easier to Develop Than You Think [Caffeine]

Java technology zone technical podcast series

Java technology zone technical podcast series: "For years, the Java zone has brought you top-quality technical content by some of the best minds in the industry. But taking the time to read an in-depth, code-heavy article can be difficult, even if it's about a topic that's critical to your day job. This new podcast series, led by the engaging and technically curious Andrew Glover, provides a new way to get information from the sources you trust most. Each week, we'll publish a new discussion with an expert on the topics that are important to your job."