Enterprise-wide Test Automation with HP Quality
Center Suite for Agile development projects.
Susan Bockhoff, McKesson
Igor Gershovich, Connected Testing
1
Agenda
 McKesson and Project Background
 Project Goals and Test Automation ROI
 Why BPT was Selected
 Test Automation Approach and Results
 Behind the scenes: the BPT architecture
 Creating Business Components
 Setting up Parameters
 Creating automated tests using BPT
Components
2
McKesson: Who Are We?
Largest healthcare services company in the world
 Fortune 15 – $106 billion in revenues (FY09)
 More than 32,000 employees dedicated to healthcare
Oldest U.S. healthcare company
 Established 1833
Only company offering solutions at every point of care
 Deep clinical, IT and process expertise
3
Horizon Clinicals Software Products
Manage information across the healthcare lifecycle
Local
Hospital
Pediatrician
Patient
Primary Care
Physician /
Medical Home
Alternative
Medicine
Practitioner
& Family
Pharmacy
Grandma’s
Physician
4
Our Software Is Complex ….
 We develop software in multiple locations
─
Westminster, Colorado
─
Alpharetta, Georgia
─
Roseville, MN
─
Bangalore, India
 Using many development tools
─
Java
─
.Net
─
GWT
─
Standard HTML
 New release contains 35 different applications and over 40,000 tests
5
Why Automate Our Tests?
 Revenue: Faster time to adoption will speed
license revenue recognition, market penetration
 Cost: Reduced cost to execute tests
 Agility: Ability to run regression tests more
quickly to speed time to market of our products.
 Repeatability: Detect regression errors more
quickly without human errors or variability.
 Coverage: Increase platform coverage on each
release.
 Increased confidence in our application quality.
6
Status at the Start of the Project
 2000 working automated tests using QTP
 These took 3 years to develop
 The team was using a keyword-driven
framework
 Tests were scripted using VB script (no record
and playback)
 Tests required a lot of maintenance and many
had to be abandoned
 There were many false failures
7
The Challenge!
 5000 new working automated tests in one year
─
Must be in production and executed successfully
─
Broad product coverage
 Be able to execute the tests in 3 days
 Tests must be re-usable over multiple releases
with a minimum of maintenance
 Automate first – SME Centric approach
 Use automated testing as part of agile
development process
8
Business Process Testing
 HP’s Business Process Testing (BPT) product
was selected for this project
 The QTP framework had some problems we
wanted to resolve:
─
Tests still required a lot of maintenance
─
Tests required automation engineers to write, and
they did not know the applications well enough to
know what the tests should do
─
When tests failed the failures could not be resolved
by the automation engineers (did not know enough
about the app to know if it was a bug)
9
Advantages of BPT
 Components are created by automation
engineers who can create them without really
knowing everything the application needs to do
 Tests themselves are created by subject matter
experts who know what the tests need to do and
how best to prioritize them.
 Test failures are analyzed by subject matter
experts who know if it is a software defect or a
problem with the automated script itself.
10
How to Select Good Candidates for
Automated Tests
 App must be in a technology that is automatable
(no apps requiring reading of bitmaps).
 Frequently run tests are automated first since
they will give the greatest payback
 This requires that a full risk-based analysis be
performed on your regression suite to prioritize
all regression tests
 High-priority regression tests (wide and shallow)
are the ones that should be automated for
highest ROI.
11
Frequency of Automated Tests
 Environmental Health Check – 1 hr. Can you log
in to all apps? Executed on every build.
 Smoke tests – 4 hrs. Can all main pages be
accessed? Executed every two weeks.
 Critical automated regression – 6 hrs. can all
major transactions be performed? Executed at
start of regression testing and also last thing
before release.
 High priority regression - executed several
times on a release.
12
Test Automation Architecture
HP Quality Center (QC)
Test
Component
Test
Test
Component
Test
Test
Component
Business Process Testing (BPT)
HP QuickTest Pro (QTP)
Application Under Test
Resources:
-Functions
-Object Rep.
-Environment
SME-centricTest
Automation Process
Automation
Engineer
Automation Architecture Design
Create Function Libraries
Create Object Repositories
Subject Matter Expert
or
Test Engineer
Create Business Components
Drag Components to create tests in QC
Configure Input/Output parameters
Debug tests in QC Test Plan module
Add tests to test set in Test Lab module
SME-centric Process Advantages
 Responsibilities are clearly defined:
─
─
Automation developers define architecture and create
components
Subject matter experts create automated test cases
 SME doesn’t need to know/use QTP.
 Uses very flexible Keyword/Data-Driven approach.
 Reduced learning curve due to same approach across
different applications/technologies
 Applicable to all areas of automated testing, like
regression, smoke, acceptance, etc.
 Designed for Maintainability
Automated Testing with Agile
Automation Engineers
Shared between Agile teams
SME Testers
Embedded in Agile teams
Create BPT components
Write automated tests using BPT
during each sprint
components during each sprint
Modify components as
needed
Execute automated tests at the end
of each sprint
Use BPT tests as regression tests
during later sprints
Benefits of New Approach
 Leverage product knowledge of SMEs to write
more meaningful and reliable tests
 Increase test coverage -> fewer product
escapes
 Ability to keep up with increased product
workload without increase in staff
 Reduce time spent analyzing automation failures
 Faster execution of automated tests
The Results
 The team created 5500 new, working BPT tests in one
year using this approach.
 With the same budget, the team increased the number of
automated tests created by 5 times using this approach.
 Automated tests now comprise 25% - 40% of the tests
executed depending on the size of the release.
 These tests are being maintained and executed
consistently.
 The team is now working on maintenance, speed of
execution, unattended execution, and new projects.
18
Automated Testing Process
Quality
Center
QTP
AUT
BPT Components,
Automation Resources
AUT
Testing
Test case
Results
Data
Step 1
SETTING UP THE BPT
ARCHITECTURE
20
Test Automation Elements
 IDE: Quality Center with BPT/Mercury QuickTest
Professional (QTP)
 Environment Files
 Application Areas
 Function Libraries
 Object Repositories
 Business Components and Test Scripts
Flexible Test Automation Lab – over
30 VMs and Desktops
Test Sets
Lab Machines
Environments Under Test
QA1
QA2
QA3
22
Environment (BPT Test) for Computer
Name–Test Environment Mapping
Computer name – Test Environment mapping
Environment.xls
Function Libraries
 Technology libraries
─
─
─
─
Web
GWT
.Net
Win32
 Common libraries
─
─
─
Utility
DB
CommProtocols
 Project Specific libraries
Step 2
CREATING REUSABLE BPT
COMPONENTS
26
Business Components
 A component is a reusable module to perform
tasks on logically independent application parts
(Page or Window).
 Components use Input/Output parameters for
data input/verification and navigation within
Page or Window.
 Input/Output parameters also are used to pass
data between components.
 Components can use iterations.
Login Component Steps
(Example for Web Application)
 Component initialization
─
Load environment variables like URL, login
credentials, DB connection strings, etc.
 Close all browsers except QC
 Invoke an application
 Login
 Verification that login is successful
Functional Component Steps
(Example for Web Application)
 Component initialization – ComponentInit() function
─
Load environment variables like URL, login
credentials, DB connection strings from
Environment.xls file using QC OTA
 Verify that Expected page is displayed
 Side or Top menu navigation
 Web Page internal flow
BPT Component Script and AUT
AUT
QTP
Step 3
CREATING AUTOMATED TESTS
BY CONFIGURING PARAMETERS
WITHIN COMPONENTS
31
BPT Test Case in Quality Center
Data
Parameter Naming Conventions
Prefix
Object
Example
lst
List
lst_User
txt
Edit
txt_User
chk
Checkbox
chk_User
rad
Radiobutton/RadioGroup
rad_User
tab
Tab
tab_User
tbl
Table
tbl_User
cell
Cell in Table
cell_User
file
WebFile
file_User
ele
WebElement
ele_User
cmb
Combobox
cmb_User
st
Static Text (Read Only)
st_User
.Apply Method
 Object.Apply Parameter(“Parameter_Name”)
.WebEdit(“Name”).Apply Parameter(“txt_Name”)
 Apply method functionality:
─
Performs data substitution
─
Verifies if object exists and visible
─
Makes decision what to do with “Name” edit object
based on Parameter data
Input Constraints for Edit (Text Box)
Object
Value
Action
Example
No action
<CLEAR>
Clear edit field
<CLEAR>
text
Enter text
Hello
[text]
Verify text
[Hello]
[<ENABLED>], [<DISABLED>]
Verify if object is
Enabled/Disabled
[<EXIST>],[<NOEXIST>]
Verify if object exist/noexist
[<REGEX>Pattern]
Verify that Pattern
matches text
[<REGEX>TestCase.*]
Working with Objects in Grids
row=WebTable().GetRowIndex(Parameter(“search_”),clmn)
WebTable().CellApply row, clmn, objType,
Parameter(“cell_”)
iRow = WebTable().GetRowIndex(Parameter(“search_Name”),2)
WebTable().CellApply iRow,3,“WebElement”,”Parameter(cell_Age”)
 Parameter prefixed with the word “search_” indicates
which row to work with.
─
Row can be selected by using row number: #n
─
the text contained in the specified Column
─
Pattern
 Parameters for objects in table cells are prefixed by
“cell_”
Example: Working with Grids
Data Substitution
 Special tags for Dates, Times, SSNs, Unique
values, DB Queries, Random, etc.
<SSN> - get substituted with unique SSN
<TODAY> - get substituted with today date (03/19/10)
<RNDDIG 3> get substituted with 3 random digits
<DateTime YYYYMMDDHHmmss>
<DB DBEnvironment::SQLStr>
 Tags can be combined
[<REGEX><TODAY> at .* AM]
Summary
 This method provides a way for test automation
engineers to create re-usable components without
having to know all the ways the application will be used.
 Subject matter experts can use the components to easily
create tests that can do anything that the application can
do without having to be software engineers.
 This method allowed us to create a great number of
working automated tests in a relatively short time.
 The project has been a major success and has helped
us achieve greater confidence in our release quality.
39
Q&A
40