Software code coverage and automated testing

I started using a tool named Cobertura to generate code coverage reports lately, and I have to say that I've been very happy with the results. If you are a believer in test driven development, or TDD, the next step in the process is code coverage.

Code coverage tools, like Cobertura, provide feedback on the lines of code that your tests actually touch. For instance, let's say you have a class named Pizza.java, and it contains 200 lines of code. Let's also say that you have a class named TestPizza.java that contains your unit tests (Junit tests in this example) for your Pizza.java class. Cobertura works by wrapping instrumentation around your Java bytecode, and then running your Junit tests for you. By doing so, Cobertura is able to see what lines in the Pizza.java class are exercised by your TestPizza.java class. Out of 200 lines of code in your Pizza class, Cobertura might come back and show that you're really only exercising something like 100 lines of code, even though you're confident that your unit tests are very thorough.

Problems uncovered

The feedback Cobertura provides is invaluable. By using Cobertura on a regular basis over the last several weeks I've found at least the following general classes of problems in my code:

  1. I discovered a lot of code that I didn't have tests for. (Cobertura makes this very obvious.)
  2. For the tests I did have, Cobertura showed that I wasn't exercising all the code I really thought I was.
  3. While working with other developers on existing projects we discovered a very significant amount of dead code.

Cobertura configuration

Cobertura is very easy to configure. The first time I used it I was able to add it to my existing Ant builds in under 30 minutes.

I may write a longer tutorial about it in the future, but for now here's a very good introduction. While I'm in the neighborhood, here is a Cobertura Ant task reference.

Code coverage tools

There are other code coverage tools available, but here's a brief list of others I've heard about:

  1. Emma: a free Java code coverage tool
  2. Atlassian Clover - Code coverage analysis