Outline • Quality in general • Software quality • Software metrics Software Quality Martin Höst What is quality? One definition of ”quality” ”The quality of a product/service is its ability to satisfy the needs and expectations of the customers” 3 1 [Bergman/Klefsjö, ”Quality”] 2 How to achieve quality? PDSA Act Plan Study Do 1 What is needed? Process-based quality Define process Develop product Improve process Top management commitment Assess product quality No Quality OK Yes Standardize process Base decisions Focus on on facts processes Focus on customer Improve continuously Everybody committed [Sommerville] Malcolm Baldridge areas 1 Leadership 2 Strategic Planning 3 Customer and Market Focus 4 Measurement, Analysis, and Knowledge Management 5 Human Resource Focus 6 Process Management 7 Business Results History • Inspector can be seen in Egyptian work of art • Acceptance sampling at Royal Mint in GB in 12th century (sample every 15th gold coin) • 1920’s: Fisher on experiments • 1931: Walter A. Shewart suggests the use of ”control charts” • Second world war: Work on acceptance sampling http://www.quality.nist.gov History (continued) • After 2nd world war: Edward Deming and Josef Juran continue work of Shewart (much work in Japan) • Work in Japan: Kaoru Ishikawa (origin of 7-QC tools), Taiichi Ohno (embryo to QC-circles): ideas of TQM • Taguchi on experiments • Malcolm Baldridge award, ISO 9000, SIQ, CMM… Software quality Development technology Process quality Product quality People quality Cost, time and schedule 2 Elements of a software quality programme • Training program • Evaluation of effectiveness of current methods • Planning standards • Change control etc. • Recording of all defects found • Root cause analysis • Usage of userfeedback • Evaluation of how standards etc. are followed • Empowerment of staff Quality assurance Organization Quality planning Project Project Quality control Practical process quality - organizational level • Define process standards such as how reviews should be conducted, configuration management, etc. • Monitor the development process to ensure that standards are being followed • Report on the process to project management and software procurer Quality planning – project level • A quality plan sets out the desired product qualities and how these are assessed and define the most significant quality attributes • It should define the quality assessment process • It should set out which organisational standards should be applied and, if necessary, define new standards ISO 9000 principles ISO 9000 International set of standards for quality management Applicable to a range of organisations from manufacturing to service industries ISO 9001 applicable to organisations which design, develop and maintain products ISO 9001 is a generic model of the quality process Must be instantiated for each organisation Project (9000:2000) • • • • • Customer focus Leadership Involvment of people Process approach System approach to management • Continual improvement • Factual approach to decision-making • Mutual beneficial supplier relationships 3 Quality plan structure • Product introduction • Product plans • Process descriptions • Quality goals • Risks and risk management Quality plans should be short, easy to understand documents. If they are too long, no-one will read them Software quality assurance plan according to IEEE 730-1989 • Purpose • Reference documents • Management • Documentation • Standards, practices, conventions, metrics • Reviews and audits • Test • Problem reporting and corrective actions • Tools techniques and methodologies • Code control • Media control • Supplier control • Records collection and maintenance • Training • Risk management [Humphrey –89] Standards • Standards are the key to effective quality management • They may be international, national, organizational or project standards • Product standards define characteristics that all components should exhibit e.g. a common programming style • Process standards define how the software process should be enacted Standards development • Involve practitioners in development. • Review standards and their usage regularly. • Detailed standards should have associated tool support. Standards Advantages: • Best practice (avoids repetition of past mistakes) • Framework for quality assurance process • Provide continuity new staff can understand the organisation Problems: • Not seen as relevant and up-to-date by software engineers • Involve too much bureaucratic form filling? • Unsupported by software tools Documentation standards Particularly important - documents are the tangible manifestation of the software •Documentation process standards – How documents should be developed, validated and maintained •Document standards – Concerned with document contents, structure, and appearance •Document interchange standards – How documents are stored and interchanged between different documentation systems 4 Quality control Document standards •Document identification standards – How documents are uniquely identified •Document structure standards – Standard structure for project documents •Document presentation standards – Define fonts and styles, use of logos, etc. •Document update standards – Define how changes from previous versions are reflected in a document • Checking the software development process to ensure that procedures and standards are being followed • Two approaches to quality control – Quality reviews – Automated software assessment and software measurement Metrics assumptions [Kitchenham] Software metrics “You cannot control what you cannot measure” Tom DeMarco, Controlling Software Projects, 1982 • Product, process or resources • Internal, external • Allows for objective comparisons between techniques and processes • Systematic use of measurement is still uncommon Products, processes, resources Examples of entities Code (product) Examples of internal attributes Examples of external attributes Size, reuse, modularity, Quality, complexity, coupling… maintainability… Design process (process) Time, effort, number of specification faults found… Cost-effectiveness Personnel (resources) Age, price Productivity, experience, intelligence… • A software property can be measured • The relationship exists between what we can measure and what we want to know • This relationship has been formalized and validated • It may be difficult to relate what can be measured to desirable quality attributes Product measurement process Choose measurements to be made Analyse anomalous components Select components to be assessed Identify anomalous measurements Measure component characteristics [Fenton] 5 Examples of product metrics Data accuracy Don’t collect unnecessary data – The questions to be answered should be decided in advance and the required data identified Tell people why the data is being collected – It should not be part of personnel evaluation Don’t rely on memory – Collect data when it is generated not after a project has finished Experimentation – learning from experience Experiment Fan-in/Fan-out Length of code Cyclomatic complexity Length of identifiers Depth of conditional nesting Fog index Reality • Most large SW-organisations have some kind of QA-programme • Working with “other” organisations requires QA • Metrics are collected • Independent variables • Dependent variables • Confounding factors IV • • • • • • DV CF – But there is a risk of getting “data cemeteries” • In late projects there is a risk of “bypassing” standards Summary • Quality is a broad term • May mean different things for different customers • Software quality assurance is based on a relationship between the process and the product • Standards are important • Measurement is important 6