Outline Software Quality What is quality? One definition of ”quality”

Outline
• Quality in general
• Software quality
• Software metrics
Software Quality
Martin Höst
What is quality?
One definition of ”quality”
”The quality of a product/service is its ability
to satisfy the needs and expectations of the
customers”
3
1
[Bergman/Klefsjö, ”Quality”]
2
How to achieve quality?
PDSA
Act
Plan
Study
Do
1
What is needed?
Process-based quality
Define process
Develop
product
Improve
process
Top management commitment
Assess product
quality
No
Quality
OK
Yes
Standardize
process
Base decisions
Focus on
on facts
processes
Focus on customer
Improve
continuously
Everybody
committed
[Sommerville]
Malcolm Baldridge areas
1 Leadership
2 Strategic Planning
3 Customer and Market Focus
4 Measurement, Analysis, and Knowledge
Management
5 Human Resource Focus
6 Process Management
7 Business Results
History
• Inspector can be seen in Egyptian work of art
• Acceptance sampling at Royal Mint in GB in 12th
century (sample every 15th gold coin)
• 1920’s: Fisher on experiments
• 1931: Walter A. Shewart suggests the use of
”control charts”
• Second world war: Work on acceptance sampling
http://www.quality.nist.gov
History (continued)
• After 2nd world war: Edward Deming and Josef
Juran continue work of Shewart (much
work in Japan)
• Work in Japan: Kaoru Ishikawa (origin of 7-QC
tools), Taiichi Ohno (embryo to QC-circles): ideas
of TQM
• Taguchi on experiments
• Malcolm Baldridge award, ISO 9000, SIQ,
CMM…
Software quality
Development technology
Process quality
Product quality
People quality
Cost, time and schedule
2
Elements of a software quality
programme
• Training program
• Evaluation of
effectiveness of
current methods
• Planning standards
• Change control etc.
• Recording of all
defects found
• Root cause analysis
• Usage of userfeedback
• Evaluation of how
standards etc. are
followed
• Empowerment of staff
Quality assurance
Organization
Quality
planning
Project
Project
Quality control
Practical process quality
- organizational level
• Define process standards such as how reviews
should be conducted, configuration management,
etc.
• Monitor the development process to ensure
that standards are being followed
• Report on the process to project management
and software procurer
Quality planning – project level
• A quality plan sets out the desired product
qualities and how these are assessed and
define the most significant quality attributes
• It should define the quality assessment
process
• It should set out which organisational
standards should be applied and, if
necessary, define new standards
ISO 9000 principles
ISO 9000
International set of standards for quality
management
Applicable to a range of organisations from
manufacturing to service industries
ISO 9001 applicable to organisations which design,
develop and maintain products
ISO 9001 is a generic model of the quality process
Must be instantiated for each organisation
Project
(9000:2000)
•
•
•
•
•
Customer focus
Leadership
Involvment of people
Process approach
System approach to
management
• Continual
improvement
• Factual approach to
decision-making
• Mutual beneficial
supplier relationships
3
Quality plan structure
• Product introduction
• Product plans
• Process descriptions
• Quality goals
• Risks and risk management
Quality plans should be short, easy to understand
documents. If they are too long, no-one will read
them
Software quality assurance plan
according to IEEE 730-1989
• Purpose
• Reference documents
• Management
• Documentation
• Standards, practices,
conventions, metrics
• Reviews and audits
• Test
• Problem reporting and
corrective actions
• Tools techniques and
methodologies
• Code control
• Media control
• Supplier control
• Records collection and
maintenance
• Training
• Risk management
[Humphrey –89]
Standards
• Standards are the key to effective quality
management
• They may be international, national, organizational
or project standards
• Product standards define characteristics that all
components should exhibit e.g. a common
programming style
• Process standards define how the software
process should be enacted
Standards development
• Involve practitioners in development.
• Review standards and their usage
regularly.
• Detailed standards should have associated
tool support.
Standards
Advantages:
• Best practice (avoids
repetition of past
mistakes)
• Framework for quality
assurance process
• Provide continuity new staff can
understand
the organisation
Problems:
• Not seen as relevant
and up-to-date by
software engineers
• Involve too much
bureaucratic form
filling?
• Unsupported by
software tools
Documentation standards
Particularly important - documents are the tangible
manifestation of the software
•Documentation process standards
– How documents should be developed, validated and
maintained
•Document standards
– Concerned with document contents, structure, and
appearance
•Document interchange standards
– How documents are stored and interchanged between
different documentation systems
4
Quality control
Document standards
•Document identification standards
– How documents are uniquely identified
•Document structure standards
– Standard structure for project documents
•Document presentation standards
– Define fonts and styles, use of logos, etc.
•Document update standards
– Define how changes from previous versions are
reflected in a document
• Checking the software development
process to ensure that procedures and
standards are being followed
• Two approaches to quality control
– Quality reviews
– Automated software assessment and software
measurement
Metrics assumptions
[Kitchenham]
Software metrics
“You cannot control what you cannot measure”
Tom DeMarco, Controlling Software Projects, 1982
• Product, process or resources
• Internal, external
• Allows for objective comparisons between
techniques and processes
• Systematic use of measurement is still uncommon
Products, processes, resources
Examples of
entities
Code (product)
Examples of internal attributes Examples of external
attributes
Size, reuse, modularity,
Quality, complexity,
coupling…
maintainability…
Design process
(process)
Time, effort, number of
specification faults found…
Cost-effectiveness
Personnel
(resources)
Age, price
Productivity, experience,
intelligence…
• A software property can be measured
• The relationship exists between what we
can measure and what we want to know
• This relationship has been formalized and
validated
• It may be difficult to relate what can be
measured to desirable quality attributes
Product measurement process
Choose
measurements
to be made
Analyse
anomalous
components
Select
components to
be assessed
Identify
anomalous
measurements
Measure
component
characteristics
[Fenton]
5
Examples of product metrics
Data accuracy
Don’t collect unnecessary data
– The questions to be answered should be decided in
advance and the required data identified
Tell people why the data is being collected
– It should not be part of personnel evaluation
Don’t rely on memory
– Collect data when it is generated not after a project has
finished
Experimentation – learning
from experience
Experiment
Fan-in/Fan-out
Length of code
Cyclomatic complexity
Length of identifiers
Depth of conditional nesting
Fog index
Reality
• Most large SW-organisations have some
kind of QA-programme
• Working with “other” organisations requires
QA
• Metrics are collected
• Independent variables
• Dependent variables
• Confounding factors
IV
•
•
•
•
•
•
DV
CF
– But there is a risk of getting “data cemeteries”
• In late projects there is a risk of “bypassing” standards
Summary
• Quality is a broad term
• May mean different things for different
customers
• Software quality assurance is based on a
relationship between the process and the
product
• Standards are important
• Measurement is important
6