Software Quality Measurement And Metrics (1) 2f1y6k

Software Quality Assurance

Software Quality Measurement and Metrics

Outline ￭ ￭ ￭ ￭ ￭ ￭

Basic Terminologies Project Indicators Classification of Software Metrics Software Quality Metrics Software Maintenance Metrics Software Defects Classification Software Quality Measurement and Metrics

2

Software Metrics “Software Metrics let you know when to laugh and when to cry” (Tom Glib)

Software Quality Measurement and Metrics

3

Basic Terminologies ￭

Measure –

￭

Measurement –

a quantitative indication of some attribute of a product or process. For example, 5 errors, 1200 LOC etc the act of determining a

measure. ￭

Metric –

a quantitative measure of the degree to which a product or process possesses a given attribute. For example, 5 defects / KLOC. Metric is, basically, a ratio of two related measures. Software Quality Measurement and Metrics

4

Why Measurement ? ￭

There are four reasons for measuring software processes, products and resources: 1. 2. 3. 4.

To To To To

Characterize Evaluate Predict Improve Software Quality Measurement and Metrics

5

Types of Measurements Direct Measures

￭ ￭

Easily and directly measurable. For example, LOC, Effort, Cost etc.

Indirect Measures

￭ ￭ ￭

￭

Not easy to measure. Indirectly measured from direct measures etc. For example, complexity, quality etc. Software Quality Measurement and Metrics

6

Indicators ￭

Indicator –

￭

Indicators are of two types:

is a metric or combination of metrics that provide insight into the software process, product or project. For example, two teams using two different review approaches give an indicator of the better approach.

￭ ￭

Process Indicators Project Indicators Software Quality Measurement and Metrics

7

Indicators Process Indicators

￭ ￭

￭

￭

Used to gain the insight into the efficacy of the existing process. These are collected across all projects and over a long periods of time. For example, PSP and TSP etc.

Project Indicators

￭ ￭ ￭ ￭ ￭ ￭

￭

Used to asses the status of the project. Track potential risks. Uncover problems areas before they occur. Adjust the work flow and tasks accordingly. Evaluate project’s team ability to control software quality. For example, EVA etc. Software Quality Measurement and Metrics

8

Classification of Metrics

Software Quality Measurement and Metrics

9

Quality Metrics ￭

￭

Provide indicators to improve the quality of the product. There are many quality attributes such as maintainability, usability, integrity and correctness etc. (see McCall’s Quality Model)

Software Quality Measurement and Metrics

10

Availability ￭

￭

Availability is concerned with system failure and its associated consequences. The availability of a system is the probability that it will be operational when it is needed. This is typically defined as:  

￭

Mean Time to Failure Mean Time to Failure + Mean Time to Repair

From this come like 99.9% availability, or a 0.1% probability that the system will not be operational when needed. Software Quality Measurement and Metrics

11

Maintainability ￭

￭

￭

￭

Maintainability is the degree of easiness with which a program can be corrected if an error is encountered or changed if requirement changed. Mean Time To Change (MTTC) is the indirect measure to measure the maintainability. MTTC is the mean time to fulfill a change request. Lower MTTC means high maintainability. Software Quality Measurement and Metrics

12

Correctness ￭

￭

Correctness is the degree to which the software performs its required function. A common way to measure the correctness is Defect Density (DD) No. of Defects reported by the DD 

Size of the program in KLOC

Software Quality Measurement and Metrics

13

Usability ￭

￭

Usability is concerned with how easy it is for the to accomplish a desired task and the kind of the system provides. It be broken down into: 1. 2. 3. 4. 5.

Learning system features Using a system efficiently Minimizing the impact of errors Adapting the system to needs Increasing confidence and satisfaction Software Quality Measurement and Metrics

14

Integrity ￭

￭

￭

Ability of the system to withstand attacks on its security. Integrity is measured in of threat and security. Threat – probability that an attack of specific type will occur within a given time.

￭

.

Security –

probability that an attack of specific type will be repelled. Integrity = ∑ [ 1 - (threat) * (1 – security) ] Software Quality Measurement and Metrics

15

Performance Performance is concerned with how long it takes the system to respond when an event occurs. The response of the system to an event can be characterized by:

￭

￭

￭ ￭

￭

￭

Latency = response time – event occurrence time. Throughput = the number of transactions the system can process in a second .

The Jitter of Response = the variation in latency The number of events not processed because the system was too busy to respond Software Quality Measurement and Metrics

16

Occurrence of Events over time

MTBF

MTTD MTTR Time Fault occurs

Error caused

Detection Of error

Repair

Error Fault occurs caused

Defect Rate 

Defect Injection Rate (DIR)

[(No of in process defects) + (No of Customer-reported defects)] Actual size of the product

Defect Removal Efficiency (DRE) Defect found by removal operation Defect Present at removal operation     

X100%

System Testing Acceptance Testing Warranty Customer reported during work product reviews Technical Reviews

Defect Removal Effectiveness No of defects removed (at the step entry)

X100% [(no of defects existing at the step entry) + (No of defects injected)]

Activities Associated with Defect Injection and Removal Development Phase

Defect Injection

Defect removal

Requirements

Requirement Requirement Analysis gathering, and and Review development of functional specification

High level Design

Design Work

Design Inspections

Low level Design

Design Work

Design Inspections

Code Implementation

Coding

Code Inspections

Integration/Build

Integration process

Build Testing

Unit Test

Bad Fixes

Testing Itself

Component Test

Bad Fixes

Testing Itself

System Test

Bad Fixes

Testing Itself

Verification

Defect Injection and Removal During One Process Step

Undetected defects Defect Defects Existing on Injected Step Entry during Development

.

Defect Detection Defect Repair

Incorrec t Repairs Defects Removed

Defect Existing

Phase based Defect Effectiveness 

High Level Design Inspection Effectiveness Defects Removed Defects existing (Escaped from requirement phase) + Defects Injected in High Level Design

Phases

Defect Injected

Defect Remov ed

Defect Remaine d in each Phase

In-Process

Customer Reporte d

Requirements

85

15

88

12

High level design

55

34

69

20

Low Level Design

23

12

30

5

Code

44

0

40

4

Unit Testing

78

52

100

30

System testing

67

23

40

50



Calculate    

Defect Defect Defect Defect phase

Density if size is 5000 KLOC Rate Removal Efficiency Removal Effectiveness of each

Software Maintenance Metrics

Some Maintenance Metrics 



 

Fix backlog and backlog management index Fix response time and fix responsiveness Percent delinquent fixes Fix quality

Fix backlog and backlog management index 

BMI

Numbers of problem closed during the month X100% Numbers of problems arrivals during the month



As a ratio if BMI is larger than 100, it means the backlog is reduced, if is less than 100 than it is increased

Fix response time and fix responsiveness 

Mean time of all problems from open to closed



MT-pr1 + MT-pr2+…….+MT-prn 

The metric is operationalized as percentage of delivered fixes meeting committed dates to customer

Percent delinquent fixes Number of fixes that exceeded the response time criteria by severity level Number of fixes delivered in a specified time 

This metric is only for closed problems

X100%

Some other measures 

Efficiency

Where Size = KLOC Effort = man year Time = Hours B = constant or a scaling factor and is a function of the project size.

REQUIREMENT VOLATILITY

Software Quality Assurance

Defects Classification

Defects Classification Error

1. ￭ ￭

Errors are human mistakes. For example, spell mistakes or syntactic errors etc.

Defects

2. ￭

￭

￭

Defects are improper program conditions that generally results from errors, but not always. For example, documentation errors do no result in defects. And Incorrect Packaging or exception handling Software Quality Measurement and Metrics

34

Defects Classification Bugs / Fault

3.

It is a program defect that is encountered in operation (alpha testing, beta testing or software operation) All defects do not cause bugs. For example, Y2K

￭

￭ ￭

Failure

4.

A failure is a malfunction of ’s installation. It may result from a

￭ ￭ ￭ ￭ ￭ ￭

Bug Incorrect installation Communication line hit Hardware failure etc. Software Quality Measurement and Metrics

35

Defects Classification Problems

5. ￭

￭

￭

Problems are encountered difficulties. Problems are human events whereas Failures are system events. A problem may result from: ￭ ￭ ￭

Failure Misuse misunderstandings Software Quality Measurement and Metrics

36

Software Quality Assurance

Feel free to ask! Thanks!