Processes improvement initiatives everywhere and not a single measurement to explain them.
How many slide decks have you seen where the cost savings shown seem more hypothetical and unrealistic than reality? I got into the habit of creating a folder in my mail inbox labeled ‘PIFMA’ which stands for ‘pulled it from my a$$’.
My rule of thumb is, if you can’t carry it forward into a working model using a standard measurement checklist (the result of answering a series of measurement review questions) then you’re either guessing, or you carry a lot of implicit knowledge and believe you are an SME, aka the informed guesser, or perhaps, you carry a lot of weight and no one challenges your numbers out of fear.
I prefer to start with good old fashioned heuristics and clear and concise KPI’s that have been baselined by them. Does it take more would than whipping up a bunch of fluff numbers? Yes! Unlike fluff numbers, is it more reliable and capable of building an economically fundable model that can withstand the test of time, scope and quality? Yes!
Here’s one metric that I have used when creating executive slide decks for CIO’s, CTO’s and COO’s to secure funding and then putting into practical use with a team to be measured by.
Defect removal costs per function point
Let’s say, I wanted to make sure a team was focused on the defect removal costs per function point and not the cost per defect metric. My conversations central theme is that I want to prove that with quality built into a product early I can substantially lower the cost per defect. How would I talk a team through this?
Assume, all defect removal operations have a significant quantity of fixed costs associated with them. It follows that, as the number of defects reported declines, the cost per defect must rise. It is reasonable to argue that cost per defect is not valid for serious economic analysis because every downstream activity will have a higher cost per defect than upstream activities. Therefore, it’s important that discussion around measurement stays focused on the right metric.
Example:
I have a software application that contains 100 function points. During each of my quality assurance processes the software will go through three consecutive test stages, each of which will test 50 percent of the function points. Writing the test cases for each test stage costs $1,000. Running the tests for each stage costs $1,000. Fixing each discovered defect costs $100. What is the economics of the three test stages?
In the first test stage, the costs were $1,000 for writing test cases, $1,000 for running test cases, and $5,000 for fixing 50 defects, or $7,000 in all. The “Defect removal costs per function point” for test stage 1 would be $70. Consider, there were 50 function points tested out of 100 total function points. This amounts to $140 per test.
In the second test stage, the costs were $1,000 for writing test cases, $1,000 for running test cases, and $2,500 for fixing 25 defects, or $4,500 in all. The “Defect removal costs per function point” for test stage 2 would be $45. Considering there are now 25 function points tested out of the remaining 50 the cost is $180 per test.
In the third test stage, the costs were $1,000 for writing test cases, $1,000 for running test cases, and $1,200 for fixing 12 defects, or $3,200 in all. The “Defect removal costs per function point” for test stage 3 would be $32. With only 12 tests remaining the cost jumps to $267 per test.
As can be seen from this example, the fixed costs of writing and running test cases tend to drive up the later per test costs at each stage even where few defects are found. However the cost of defect removal per function point decreases.
That is a metric you should consider for measurement. Granted, it’s only one – there are and can be many more – but, this is where you can start the conversation that drives home the main point of your discussions central theme.
Note: One could argue that defect cost does not rise because resource costs are fixed. Therefore, defect removal costs remain the same no matter which environment or stage the defect was found in. Considering how many more resources are involved in the removal of a defect as it migrates through your ecosystem and not forgetting the potentially irreparable damage caused by defects found by your customers it’s not as easy to dismiss the metric I am proposing. Granted, you should always be rigorous in your metric assumptions, consider other models, calculations and provide real examples that apply to your projections. The design of your experiment must be suited to your situation.
qualitasrebus 