Test Automation: An
Architected Approach
Dan Young
March 17th, 2005
[email protected]
What goes wrong?
Vendors push record/playback as a
viable solution
Managers want to believe the hype
and believe that they can see results
immediately
Users under pressure rush for any
implementation that works
Planning an Automation Project
Requirements
Balancing Personnel
An Automation Plan
Return on Investment
An Architected Solution
Encapsulated functions (functions
within functions)
Encapsulated test cases
Managing Results
Pre-scripting
An Architected Solution:
Encapsulated Non-Scripted
Functions
Low-level “nonscripted” functions
handle common
processes that a business
analyst isn’t necessarily
concerned with.
May accept parameters
May return a value
Recursion may be used
at this level
An Architected Solution:
Encapsulated Scripted Functions
Handle the entire process of what
a test case does
Accept parameters, and parse
variables as appropriate
Most likely do not return a value
Do write a result
An Architected Solution:
Sample Application
Contact
Management
software
Sample
Ability: Add
New Contact
An Architected Solution:
Encapsulated Scripted Functions II
The first step is to parse the parameters
passed from the test case
Individual values are assigned to
variables for later use
An Architected Solution:
Encapsulated Scripted Functions III
Function then performs
the entire test case,
including launching the
app, selecting the
database, populating the
fields, verification and
cleanup
Errors (expected or
unexpected) are handled
within the function
An Architected Solution:
Encapsulated Test cases
Test case focuses on a
particular area of
functionality
Test case consists of
initializing variables and
making a single
function call.
Anyone can add test
cases with very little
training, or even
knowledge of the AUT
An Architected Solution:
Managing Results: Flat File Model
All results written to an
external txt or csv file
Results written to a
particular directory structure
for ease of comparison by a
third party comparison
utility
Only files that fail
comparison need to be
examined
Root directory of results
directory structure is unique
so that results of multiple
test runs can be captured
An Architected Solution:
Managing Results: Database Model
Automation tool can write results directly to a database.
Queries and Reports can be designed to quickly show
test status
Could be used to populate a web page with up-to-theminute test results.
An Architected Solution:
Pre-Scripting
With simplified test
case entry, all fields
and scripted
functions can be
agreed upon in
advance
Test cases can be set
up before functions
are written
Selling the Architected Solution
Management is under pressure to get results
quickly, but good automation architecture
requires planning and time for infrastructure
New projects: Balance the need for quick
results with the need for a maintainable
architecture
Existing projects: Architecture can be built in
as automation for particular areas of the
product needs revised
Selling the Architected Solution:
Benefits
Long-term cost and risk reduction since a modular approach
minimizes maintenance, and a lot of the infrastructure is
reusable between projects and products
Consistent levels of automated testing across projects
Provides more defined career paths for QA staff
Increased employee satisfaction and productivity since an
employee can focus on areas of interest
Anyone with product knowledge can begin entering test cases
quickly
Automation comes on line more quickly since manual testing
is done simultaneously
Readily available results save time for development as well as
QA
Selling the Architected Solution:
Pitfalls to Avoid
Include all necessary people in planning
(e.g. project manager, product manager,
testing lead, system architect). Each role
may have information that can be designed
into the initial architecture instead of added
later
Avoid the temptation to shortcut the process
Try to have some idea of what comes after
the current project
Selling the Architected Solution:
Pitfalls to Avoid II
Do not over-encapsulate
Conclusion
Properly planned and implemented
automated testing will reduce project cost
and risk in the long term
Good planning and management support are
imperative for maintainable, lasting
automation
Questions??