The Mindset

In a lecture I gave today about unit testing, I was asked what the rationale for writing unit tests is: wouldn’t it be better if developers used the same automated testing environment used by the QC team to continuously test their code?

At first, this seems like a logical proposition. If we already have an automated testing environment, why shouldn’t developers use it as well instead of writing their own tests?

Here are a few reasons for sticking with unit testing:

QC Automation Availability

To start with, the tests executed by QC are not always fully automated. In many cases, the QC staff still does a lot of manual work. For developers this repetitive manual work would be a test-killer: there is no chance that developers will run the tests for every change they do in the code if it involves a time consuming and tedious work.

Even if the QC team in your project will have a fully automated testing environment, it might take quite some time until such an environment is stable and ready for use. This means that for a substantial part of the time you won’t be able to use this environment.

Granularity of Tests

Tests performed by the QC team treat the component you write as a black box. Unit tests, on the other hand, have a much finer resolution. They test the code from within and thus making it much simper to locate bugs in the code.

When a test written by a developer on the internals of her code fails, the developer can immediately trace the relevant flawed code. The reason is that each test is designed to verify a relatively small scope of the code (in some cases, even a single method). A black box test executed on the component externally, might fail due to numerous reasons. This might require the developer to perform an extensive analysis of the test execution in order to locate the defect in her code.

Controlling the Tests

Unit tests (and the frameworks that support them) enable great control on the test environment. Causing a connection to a database to throw an exception in a production environment is almost impossible. Achieving that using a mock database in your unit-testing environment is a piece of cake. The same applies to numerous peripheral devices and components your code needs to interact with.

A well-designed unit testing environment enable the developer to easily create any scenario her code needs to deal with. A black-box production-oriented testing environment as used in QC might not always stand up to this challenge.

Environment Complexity

On a similar note, the testing environment used by the QC team is often too complex to be used extensively by developers. A “simple” testing environment might involve interaction with other components (or simulators), a database, the file system, and other peripherals. Setting up such an environment and properly configurating it might require quite some time and effort.

In many cases, the challenge is even greater. Let’s say all the developers are working on a single Linux machine. Replicating a production-oriented testing environment for each of the developers might introduce some difficulties (separate database for each developer, different ports assigned to each of them, etc).

A unit-testing environment, on the other hand, is self-contained. Peripherals are emulated using mock objects internally. Setup, configuration and potential collisions all become non-issues in such a simple environment.

Better Design

Last but not least, designing and writing unit tests significantly improves the design of the tested code. In order to create a good unit-testing environment, you must think about your interfaces and abstractions, and make your code flexible and modular. This is a great side-benefit.

The tests performed by QC treat your code as a black box. They will pass (or fail) regardless of the internal design of your code. Therefore, in the best-case scenario, you will know your code executes as expected. You will not know that it is well designed.

*** 

So, if you are a developer, my advice to you is to stick to unit testing, even if your QC team has an automated testing environment. The benefits of unit testing will enable you to always gain an immediate feedback on the functional quality of your code. Of course, it won’t hurt to use external box automated tests as well from time to time (assuming they exists), but such tests cannot replace the benefits of a well designed unit testing environment.