Test Metrics

     Test Metrics is most important to measure the quality, cost and effectiveness of the project and the processes. Without measuring these, project can’t be completed successfully. Test metrics helps,

• To take the decision for next phase of activities such as, estimate the cost & schedule of future projects.
• To understand the kind of improvement required to success the project.
• Take decision on process or technology to be modified.

Different Types of Test Metrics

Base Metrics (Direct Measure)

    Base metrics constitute the raw data gathered by a Test Analyst throughout the testing effort. These metrics are used to provide project status reports to the Test Lead and Project Manager; they also feed into the formulas used to derive Calculated Metrics.

Base Metrics	Test Phases
# of Test Cases	Test Development Phase
# of Test Cases Executed	Test Execution Phase
# of Test Cases Passed	Test Execution Phase
# of Test Cases Failed	Test Execution Phase
# of Test Cases Under Investigation	Test Development Phase
# of Test Cases Blocked	Test dev/execution Phase
# of Test Cases Re-executed	Regression Phase
# of First Run Failures	Test Execution Phase
Total Executions	Test Reporting Phase
Total Passes	Test Reporting Phase
Total Failures	Test Reporting Phase
Test Case Execution Time	Test Reporting Phase

Calculated Metrics (Indirect Measure)

    Calculated Metrics convert the Base Metrics data into more useful information. These types of metrics are generally the responsibility of the Test Lead and can be tracked at many different levels (by module, tester, or project).

Calculated Metrics	Test Phases
% Complete	Test Reporting Phase or Post test Analysis phase
% Defects Corrected	Test Reporting Phase or Post test Analysis phase
% Test Coverage	Test Reporting Phase or Post test Analysis phase
% Rework	Test Reporting Phase or Post test Analysis phase
% Test Cases Passed	Test Reporting Phase or Post test Analysis phase
% Test Effectiveness	Test Reporting Phase or Post test Analysis phase
% Test Cases Blocked	Test Reporting Phase or Post test Analysis phase
% Test Efficiency	Test Reporting Phase or Post test Analysis phase
1st Run Fail Rate	Test Reporting Phase or Post test Analysis phase
Defect Discovery Rate	Test Reporting Phase or Post test Analysis phase
Overall Fail Rate	Test Reporting Phase or Post test Analysis phase

Formulas for Calculating Metrics

1. %ge Test cases Executed: This metric is used to obtain the execution status of the test cases in terms of %ge.

%ge Test cases Executed = (No. of Test cases executed / Total no. of Test cases written) * 100.

2. %ge Test cases not executed: This metric is used to obtain the pending execution status of the test cases in terms of %ge.

%ge Test cases not executed = (No. of Test cases not executed / Total no. of Test cases written) * 100.

3. %ge Test cases Passed: This metric is used to obtain the Pass %ge of the executed test cases.

%ge Test cases Passed = (No. of Test cases Passed / Total no. of Test cases Executed) * 100.

4. %ge Test cases Failed: This metric is used to obtain the Fail %ge of the executed test cases.

%ge Test cases Failed = (No. of Test cases Failed / Total no. of Test cases Executed) * 100.

5. %ge Test cases Blocked: This metric is used to obtain the blocked %ge of the executed test cases. A detailed report can be submitted by specifying the actual reason of blocking the test cases.

%ge Test cases Blocked = (No. of Test cases Blocked / Total no. of Test cases Executed) * 100

6. Defect Density is a measure showing the ratio of defects. against the size of a development. Size is typically expressed in terms of Function Points or Impact Points

Defect Density = No. of Defects identified / size

(Here ‘Size’ is considered as requirement. Hence here the Defect Density is calculated as number of defects identified per requirement).

7. DRE (Defect Removal Efficiency) is used to identify the test effectiveness of the software.

Defect Removal Efficiency (DRE) = (No. of Defects found during QA testing / (No. of Defects found during QA testing + No. of Defects found by End user)) * 100

8. Defect Leakage: Defect Leakage is the Metric which is used to identify the efficiency of the QA testing i.e., how many defects are missed / slipped during the QA testing.

Defect Leakage = (No. of Defects found in UAT / No. of Defects found in QA testing.) * 100

9. Defects by Severity/Priority: This metric is used to identify the no. of defects identified based on the Severity / Priority of the defect which is used to decide the quality of the software.

• %ge Critical Defects = No. of Critical Defects identified / Total no. of Defects identified * 100
• %ge High Impact Defects = No. of High Impact Defects identified / Total no. of Defects identified * 100
• %ge Medium Impact Defects = No. of Medium Impact Defects identified / Total no. of Defects identified * 100
• %ge Low Impact Defects = No. of Low Impact Defects identified / Total no. of Defects identified * 100

• %ge High Priority Defects = No. of High Priority Defects identified / Total no. of Defects identified * 100
• %ge Medium Priority Defects = No. of Medium Priority Defects identified / Total no. of Defects identified * 100
• %ge Low Priority Defects = No. of Low Priority Defects identified / Total no. of Defects identified * 100