Disciplined Test Process Drives SDLC
By Sujatha C
Disciplined Test Process
www.stagsoftware.com
1
Copyright© 2007-2008, STAG software private limited
What is planned?
• Part-1 (practical experience)
– Impact of documentation not in synch/ vague requirements
– Effective change management
– Role of SCM
• Part-2 (good practices recommended)
–
–
–
–
Where we normally face problem?
Improve incoming quality of build to QA
Disciplined approach to defect removal - Levels of testing
Test process ad-hoc to disciplined
Disciplined Test Process
www.stagsoftware.com
2
Copyright© 2007-2008, STAG software private limited
1
Part-1
Three main issues
• Requirements not captured properly
• Change requests not documented
– Impact analysis of CR’s do not happen
• Release management not in place
– Version control of work products not
happening
Disciplined Test Process
www.stagsoftware.com
3
Copyright© 2007-2008, STAG software private limited
Testing & process go handin-hand
• Testing with out proper requirements will
not uncover all defects
• Testing without change request impact
analysis will lead to defect escapes
• Testing completed but releasing the
wrong product version to the customer
will lead to severe problems
Disciplined Test Process
www.stagsoftware.com
4
Copyright© 2007-2008, STAG software private limited
2
Process can be customized
• Requirements can be captured as simple
bullets or in an excel sheet
• Requirements need to be reviewed with
customer and internally with the testing
team
• Attempt to freeze requirements before
proceeding with other steps in the SDLC
(75-80%)
Disciplined Test Process
www.stagsoftware.com
5
Copyright© 2007-2008, STAG software private limited
Process to be monitored
• Form a SEPG group
• Group to consist of Sr. and Jr. engineers
• Goal for this group
– To implement the process
– To ensure it is followed rigorously
– Conduct planned and surprise audits
Disciplined Test Process
www.stagsoftware.com
6
Copyright© 2007-2008, STAG software private limited
3
Process can be evolved
• All users must feel comfortable with the
process steps
• Adherence will become natural
• All process users (employees)
input/feedback is a must to freeze the
process steps
• A process handbook with templates and
samples can be formulated
Disciplined Test Process
www.stagsoftware.com
7
Copyright© 2007-2008, STAG software private limited
Training
• Educate the users about following the
process thru out the SDLC
• Importance of levels of testing and about
defects at different levels
• Share the product quality success with the
users to motivate them.
Disciplined Test Process
www.stagsoftware.com
8
Copyright© 2007-2008, STAG software private limited
4
Part-2
Where we normally face problem?
A cycle of test involves development of set of
requirements and tested as planned
Development effort
Planned
Test
effort
What is going wrong?
Disciplined Test Process
Actual
Test
effort
Development effort
www.stagsoftware.com
9
Copyright© 2007-2008, STAG software private limited
Improved incoming quality
Control incoming quality to test cycle
• Implement Basic Acceptance Test Scripts
• Define scope for BATS involving development team
• Set a goal to control BATS failure
• Introduce root cause analysis process
10
% Failed TC
Successful
BATS
Suggested tool
9
FAILED BATS
8
7
Limit
R1
R2
R3
R4
6
5
4
3
2
1
0
C1
Disciplined Test Process
C2
www.stagsoftware.com
10
Copyright© 2007-2008, STAG software private limited
5
A disciplined approach to defect removal
System
System functionality
Constraints
Integration
Unit
Interface issues
Resource contention issues
Performance
Data validation
Error handling
Basic functionality
Resource issues
Timing issues
Basic performance
…defects are categorized into different types
… different defect types are expected to be detected at different stages i.e. different
levels of testing detect different types of defects
… the impact of the defect increases at every level, hence this approach ensures that
defects escapes are minimized
Copyright© 2007-2008, STAG software private limited
Test process – Ad-hoc to
disciplined
Check yourself
• Do you know how many system requirements you need
to test?
• Do you know what non-functional quality factors we
need to test?
• Do you know how much time it will take to test your
product?
• Do you know how much to re-test because of changes
done to code?
• Do you know customers priority on system
requirements?
Disciplined Test Process
www.stagsoftware.com
12
Copyright© 2007-2008, STAG software private limited
6
Does this help?
Suggested template
Req # Short
description
Priority
# TS
#TC
Interface
with req.
Add required columns for tracking project status
Sort as per customer priority
All reports with respect to customer priority
Estimation accuracy improves
Disciplined Test Process
www.stagsoftware.com
13
Copyright© 2007-2008, STAG software private limited
Case study
Company: A software Product Company developing ERP solution to meet the
demands of small and medium scale Pharmaceutical, Chemical and Food
processing industries
Product: ERP solution complies with Current Good Manufacturing Practices
(cGMP) and requirements of International Regulatory Bodies such as US
FDA, EDQM, TGA, MHRA, MCC, etc.
The major components of the system are
– Human resource and payroll management
– Financial Accounting and Management
– Material Management
– Production planning
– Quality Management
– Plant Maintenance
– Sales and Distribution
Disciplined Test Process
www.stagsoftware.com
14
Copyright© 2007-2008, STAG software private limited
7
STAG contribution
What we implemented
Understood requirements by interacting with functional experts
By using our techniques
Designed module level test cases and executed
Designed module integration level test cases and executed
Designed system level end-to-end business scenarios for regression testing
Participated in multiple cycles of testing in phased delivery model
Imparted training on how to capture requirements, about importance of
Unit level testing, etc.
Introduced SDLC and testing process
Significantly helped and participated in organization SDLC process
improvement activities
Played a major role being part of SEPG (Software Engineering Process
Group) team and thus ensured that Process and testing produced a quality
product
Disciplined Test Process
www.stagsoftware.com
15
Copyright© 2007-2008, STAG software private limited
First release after
process implementation!
Defect Arrival and Closure Trend
1300
1200
1100
1000
Defects
900
800
Opened
700
Closed
600
Cum-Opened
Cum-Closed
500
400
300
200
100
0
Sep
Oct
Nov
Dec
Jan
Feb
Months
Disciplined Test Process
www.stagsoftware.com
16
Copyright© 2007-2008, STAG software private limited
8
Tangible benefits
• Using well documented test cases estimation for testing
effort improved
• Schedule variance brought under control to 5%
• Unit level defect escapes to system level reduced
drastically
• Defect escapes to the customer came down drastically
• Product quality results shared with developers and
testers helped boost their confidence
• Process followers were convinced that disciplined
process and testing will lead to quality product every
time!
Disciplined Test Process
www.stagsoftware.com
17
Copyright© 2007-2008, STAG software private limited
Thank You
Questions?
Thank You
Disciplined Test Process
www.stagsoftware.com
18
Copyright© 2007-2008, STAG software private limited
9