This article deals with the Product Quality Measures for testing
  • Customer satisfaction index
  • Delivered defect quantities
  • Responsiveness (turnaround time) to users
  • Product volatility
  • Defect ratios
  • Defect removal efficiency
  • Complexity of delivered product
  • Test coverage
  •  Cost of defects
  • Costs of quality activities
  • Re-work
  • Reliability and Metrics for Evaluating Application System Testing.

     Customer satisfaction index            

  • This index is surveyed before product delivery and after product delivery.The following are analyzed:
  •  Number of system enhancement requests per year
  • Number of maintenance fix requests per year
  • User friendliness: call volume to customer service hotline
  • User friendliness: training time per new user
  • Number of product recalls or fix releases (software vendors)
  • Number of production re-runs (in-house information systems groups)
Delivered defect quantities

They are normalized per function point (or per LOC) at product delivery (first 3 months or first year of operation) or Ongoing (per year of operation) by level of severity, by category or cause, e.g.: requirements defect, design defect, code defect, documentation/on-line help defect, defect introduced by fixes, etc.

Responsiveness (turnaround time) to users
  • Turnaround time for defect fixes, by level of severity
  • Time for minor vs. major enhancements; actual vs. planned elapsed time
Product volatility

Ratio of maintenance fixes (to repair the system & bring it into compliance with specifications), vs. enhancement requests (requests by users to enhance or change functionality)

Defect ratios
  • Defects found after product delivery per function point.
  • Defects found after product delivery per LOC
  •  Pre-delivery defects: annual post-delivery defects
  •  Defects per function point of the system modifications
Defect removal efficiency
  • Number of post-release defects (found by clients in field operation), categorized by level of severity
  • Ratio of defects found internally prior to release (via inspections and testing), as a percentage of all defects
  •  All defects include defects found internally plus externally (by customers) in the first year after product delivery
Complexity of delivered product
  •  McCabe's cyclomatic complexity counts across the system
  •  Halstead’s measure
  •  Card's design complexity measures
  •  Predicted defects and maintenance costs, based on complexity measures
Test coverage
  •  Breadth of functional coverage
  •  Percentage of paths, branches or conditions that were actually tested
  •  Percentage by criticality level: perceived level of risk of paths
  •  The ratio of the number of detected faults to the number of predicted faults.
 Cost of defects
  • Business losses per defect that occurs during operation
  • Business interruption costs; costs of work-around
  •  Lost sales and lost goodwill
  •  Litigation costs resulting from defects
  •  Annual maintenance cost (per function point)
  •  Annual operating cost (per function point)
  •   Measurable damage to your boss's career
 Costs of quality activities
  •  Costs of reviews, inspections and preventive measures
  • Costs of test planning and preparation
  • Costs of test execution, defect tracking, version and change control
  • Costs of diagnostics, debugging and fixing
  • Costs of tools and tool support
  • Costs of test case library maintenance
  • Costs of testing & QA education associated with the product
  •  Costs of monitoring and oversight by the QA organization (if separate from the   development and test organizations) 
  •   Re-work effort (hours, as a percentage of the original coding hours)
  •   Re-worked LOC (source lines of code, as a percentage of the total delivered        LOC)
  •    Re-worked software components (as a percentage of the total delivered             components)
  •  Availability (percentage of time a system is available, versus the time the system is needed to be available)
  •  Mean time between failure (MTBF).
  •  Man time to repair (MTTR)
  •  Reliability ratio (MTBF / MTTR)
  •  Number of product recalls or fix releases
  • Number of production re-runs as a ratio of production runs



  Modified On Aug-28-2018 09:44:34 AM

Leave Comment