At our last Agile Iowa meeting on Thursday, Jared Richardson, he was talking about Agile Testing Strategies. He mentioned a metric he used that I found very interesting. He took three variables and created a report that listed the top 10 method/assemblies that were at risk.
Here is the (simple) equation:
Cyclomatic Complexity (CC):
Usage (U): How many other methods/classes rely on this code.
Test Coverage Percentage (TC): The percentage of the code that is exercised by a unit test
Risk Factor = CC x U x (1 - TC)
Let's look at a couple examples:
This first table shows a method used by 50 other methods (Usage) with a Cyclomatic complexity (CC) of 20. I then vary the code coverage for this method. Without any unit tests, the risk factor is a cool "g" (or Grand). As the test coverage increases the Risk Factor decreases. When the test coverage reaches 100%, the Risk Factor is reduced to zero. The function is linear, so decreasing the usage or the complexity will also reduce the Risk Factor. So it make sense that if a method isn't used, it has a risk factor of zero.
| CC | Usage | Code Coverage | Risk Factor |
| 20 | 50 | 0 | 1,000 |
| 20 | 50 | 0.1 | 900 |
| 20 | 50 | 0.5 | 500 |
| 20 | 50 | 0.9 | 100 |
The following methods all have the same risk factor of 250. Do you think this is reasonable? Should one of these factors be weighted heavier?
| CC | Usage | Code Coverage | Risk Factor |
| 10 | 50 | .5 | 250 |
| 50 | 10 | 0.5 | 250 |
| 50 | 50 | 0.9 | 250 |
For now I'm going to see how I can get the usage and the code coverage numbers automated... I can tweak the algorithm more later.
posted @ Wednesday, June 20, 2007 10:19 PM