Developing and Implementing Verification
Requirement Definitions, the “How to” of
Verifying Requirements
Kevin Vipavetz
Senior Systems Engineer
Langley Research Center
Email: [email protected]
Phone 757-864-3817
HRA 2013 Annual Regional INCOSE Conference :
Systems Engineering and Systems Thinking – Back to Basics
Copyright © 2013 by Kevin Vipavetz LaRC, NASA
Permission granted to INCOSE to publish and use.
Agenda
•
•
•
•
•
•
•
•
•
Success Factors
Requirement Owners
Requirements Flow
Verification Matrix
Verification Sequence
Developing your Verification Requirement Definition Sheet
Compliance Statement
Configuration Control
Summary
HRA 2013 Annual Regional INCOSE Conference :
Systems Engineering and Systems Thinking – Back to Basics
Copyright © 2013 by Kevin Vipavetz LaRC, NASA
Permission granted to INCOSE to publish and use.
2
Key Verification Success Factors
• Implementing system engineering best practices
• Using common/consistent formats and processes for requirement and
verification development
• Start with good requirements
• Having accountability (ownership).
• Use a good requirements management tool
• Good configuration control of requirements and all verification artifacts
HRA 2013 Annual Regional INCOSE Conference :
Systems Engineering and Systems Thinking – Back to Basics
Copyright © 2013 by Kevin Vipavetz LaRC, NASA
Permission granted to INCOSE to publish and use.
3
Independent Reviews
Positive Lesson –
Independent reviews contributed extensively to the success of the mission
• Independent reviews provided a special oversight and help ensure verification
processes are followed
• Independent reviews provided an additional check that requirements are complete,
well communicated, and reduce the possibility of missing requirements
• Independent reviews helped enforce issues to closure
• Independent reviews are interested in your success and provide in-depth detailed
reports in areas you may not have considered
• Recommend using independent reviews extensively for requirements and verification
development
HRA 2013 Annual Regional INCOSE Conference :
Systems Engineering and Systems Thinking – Back to Basics
Copyright © 2013 by Kevin Vipavetz LaRC, NASA
Permission granted to INCOSE to publish and use.
4
Good Systems Engineering
• One source is the NPR 7123.1B “NASA Systems Engineering Processes and
Requirements” and is a good outline for Systems Engineering Management Plan
(SEMP)
• The SEMP determines how you will conduct your engineering processes for the
project
– Every project should have one
– Outlines Verification Plan content
• Also, an excellent source for good requirements management is Langley’s LMS-CP5526 “Product Requirements Development and Management”
You can email me if you would like a copy of these: [email protected]
HRA 2013 Annual Regional INCOSE Conference :
Systems Engineering and Systems Thinking – Back to Basics
Copyright © 2013 by Kevin Vipavetz LaRC, NASA
Permission granted to INCOSE to publish and use.
5
Good Requirements
• Good requirements are critical to having successful verifications
• Use skilled requirement writers
• Requirements should be:
–
–
–
–
–
–
–
–
Product oriented
Concise, using the “Who shall do What “ format
Single statement
Measureable
Feasible
Verifiable
Contain rationales (key to defining good verification activities and success criteria)
Traceable
HRA 2013 Annual Regional INCOSE Conference :
Systems Engineering and Systems Thinking – Back to Basics
Copyright © 2013 by Kevin Vipavetz LaRC, NASA
Permission granted to INCOSE to publish and use.
6
Requirement Owners
•
Requirement owners are key to getting the good requirements developed
and verified
–
They are not the requirement manager
–
They are part of the whole project life cycle
–
•
They are accountable for developing, verifying, and implementing the
requirements
o Accountability is not the same as responsibility
o All are responsible, but only one is accountable
Requirement owners will define the verification activities, success criteria,
develop compliance reports, and start the signature process for verification
approval
HRA 2013 Annual Regional INCOSE Conference :
Systems Engineering and Systems Thinking – Back to Basics
Copyright © 2013 by Kevin Vipavetz LaRC, NASA
Permission granted to INCOSE to publish and use.
7
Requirement Owners cont…
• There should be a requirement owner assigned to all requirements
• For each requirement there should be a corresponding verification
requirement
Note: Spend some time on how you want to resource load your
requirement owners by how you want to verify your requirements.
Distribute the work load.
HRA 2013 Annual Regional INCOSE Conference :
Systems Engineering and Systems Thinking – Back to Basics
Copyright © 2013 by Kevin Vipavetz LaRC, NASA
Permission granted to INCOSE to publish and use.
8
Requirements Flow and Levels
Requirement Flow
Ares I-X Flight Test Plan (Primary
and Secondary Objectives)
Ares I-X Flight
Test Vehicle
System
Requirements
L1
Verification Flow
L2
First Stage
Element
Requirements
Upper Stage
Element
Requirements
Roll Control
Element
Requirements
Crew
Module/Launch
Abort System
Element
Requirements
Avionics
Element
Requirements
First Stage
End Item Spec
Upper Stage
End Item Spec
Roll Control
End Item Spec
Crew
Module/Launch
Abort System
End Item Spec
Avionics End
Item Spec
HRA 2013 Annual Regional INCOSE Conference :
Systems Engineering and Systems Thinking – Back to Basics
Copyright © 2013 by Kevin Vipavetz LaRC, NASA
Permission granted to INCOSE to publish and use.
L3
L4
9
Requirement Flow Example
FTP
FTV SRD
FS ERD
Propulsion End
Item Spec
• Starting with: Flight Test Plan Primary and Secondary Objectives P1-P5, S1-S7
• Specific Flight Test Plan Constraint 5.1f states, “ To minimize development costs, booster avionics components
should be primarily Space Shuttle RSRM heritage whenever possible.”
•FTV – 097: The FTV shall use the Space Shuttle Solid Rocket Booster (SRB) heritage Thrust Vector Control (TVC) System.
•[Trace: P1-P5, S1-S7, FTP Constraint 5.1f]
•[Allocation: FS, Avionics]
•FS – 005: The First Stage shall utilize the SRB heritage TVC subsystem designed in accordance with specification 10CEI-0001.
•[Trace: FTV - 097]
•[Allocation: Propulsion]
• Specification 10CEI-0001 “ Part I Integrated Solid Rocket Booster for Operational Flights”
• [Trace: FS -005]
• [Allocation: N/A (reached the bottom of the product hierarchy)]
HRA 2013 Annual Regional INCOSE Conference :
Systems Engineering and Systems Thinking – Back to Basics
Copyright © 2013 by Kevin Vipavetz LaRC, NASA
Permission granted to INCOSE to publish and use.
10
Verification Compliance Matrix (VCM)
• VCM is useful for tracking and reporting verification status
for reviews
Verification
Verification
Verification Meets Success
Requirement
Number
Method
Criteria
Title
VR-FTV014
True Heading
Analysis
Yes
Closure
Artifact
No./Title
Result Summary
Applicable
Documents
Models show the
Ares I-X
nominal heading AI1-SYS-CAP, Ares-I-X
Similitude
errors throughout Guidance, Navigation,
Trajectory: AI1- the flight stay less and Control System
TAR-FTVTRAJ than 1 degree for
Design Document
all analysis tools.
HRA 2013 Annual Regional INCOSE Conference :
Systems Engineering and Systems Thinking – Back to Basics
NonRequirement Owner
conformance
None
Vipavetz
Copyright © 2013 by Kevin Vipavetz LaRC, NASA
Permission granted to INCOSE to publish and use.
11
Verification Sequence
HRA 2013 Annual Regional INCOSE Conference :
Systems Engineering and Systems Thinking – Back to Basics
Copyright © 2013 by Kevin Vipavetz LaRC, NASA
Permission granted to INCOSE to publish and use.
12
Requirement Example
1.1.1.1

FTV-014: True Heading [Baseline]
The FTV shall maintain a 90 +/- 2 degrees true heading angle from pitch over to initiation
of separation
[Rationale: Given that the Ares I-X mission exercises the first stage burn only, achieving
approximately Mach 4.5, there are no orbital parameter requirements. Thus, the FTV will
need to follow a “due East” profile over the Atlantic Ocean upon launch. This sets up FS
recovery that meets P3 Test Objective.]
[Trace: FTP Constraint 5.1e, P1, P3]
[Requirement Owner] Kevin Vipavetz (Vip)
[Verification Method] Analysis
[Allocation: FS, Avionics]
[Priority: 2]
HRA 2013 Annual Regional INCOSE Conference :
Systems Engineering and Systems Thinking – Back to Basics
Copyright © 2013 by Kevin Vipavetz LaRC, NASA
Permission granted to INCOSE to publish and use.
13
Developing Verification Activity and
Success Criteria
•
•
•
•
The requirement owner is the one who fills out the Verification Definition Sheets
and work the activities and criteria
The requirement rationale and performance or function of the requirement will
help determine what your verification activity and success criteria will be
If you are having trouble with this part of the verification then most likely you have
a poor requirement written
VR-FTV-014 activity will use a comparison of models for analysis and the success
criteria is the heading must stay under +/- 2 degrees for all the models
HRA 2013 Annual Regional INCOSE Conference :
Systems Engineering and Systems Thinking – Back to Basics
Copyright © 2013 by Kevin Vipavetz LaRC, NASA
Permission granted to INCOSE to publish and use.
14
Verification Requirement Definition
Sheet (VRDS)
• Defines how you do the verification process for each
requirement
• It contains the verification implementation and closeout
information
HRA 2013 Annual Regional INCOSE Conference :
Systems Engineering and Systems Thinking – Back to Basics
Copyright © 2013 by Kevin Vipavetz LaRC, NASA
Permission granted to INCOSE to publish and use.
15
VRDS Terms Top Portion
• Verification Implementation section
–
Verification number: Unique ID linked to corresponding requirement
–
Requirement Title: Name of you verification requirement
–
Verification Method: Inspection, Demonstration, Analysis or Test
–
Verification Activity: Description of how the verification is carried out
–
Success Criteria: What was the criteria used to know we are compliant
–
Rationale: Why the this verification method was chosen
HRA 2013 Annual Regional INCOSE Conference :
Systems Engineering and Systems Thinking – Back to Basics
Copyright © 2013 by Kevin Vipavetz LaRC, NASA
Permission granted to INCOSE to publish and use.
16
Ares I–X FTV System
Verification Definition Sheet
FTV SRD Requirement to be Verified
Verification Number:
Requirement Number:
Priority (1-3):
VR-FTV-014
FTV-014
2
Requirement Title: True Heading
Verification Implementation
Verification method: The maintenance of the true heading angle defined in the AI1-SYS-SRD shall be verified by
Analysis.
Description of verification activities to be performed: The analysis shall be done using a NASA approved 6-DOF
non-linear flight dynamics model of the integrated vehicle. The analysis shall be done using Monte Carlo techniques
with an ensemble of input values that model up to 3-sigma dispersions for vehicle performance and environmental
conditions.
Success Criterion: The verification shall be considered successful when the analysis shows that the true heading
angle is less than +/- 2 degrees during flight (from pitch over to initiation of separation) with a probability of at least
95%.
Rationale: It is not possible to adequately test the flight dynamical behavior of the FTV in a ground-based
environment prior to launch, thus analysis must be used.
HRA 2013 Annual Regional INCOSE Conference :
Systems Engineering and Systems Thinking – Back to Basics
Copyright © 2013 by Kevin Vipavetz LaRC, NASA
Permission granted to INCOSE to publish and use.
17
VRDS Terms Bottom Portion
• Verification Closeout
–
Non-Conformance reports: Deviations, waivers, or failures
–
Closure Report number: Unique ID tracking number
–
Applicable Documents: Procedures, drawings, manuals etc. used for the verification
–
Verification Report: Final closeout report
–
Event preceding verification activity: Used to assist scheduling
–
Estimated Duration of Verification Activity: Used to assist scheduling
–
Verification Closure Date: Date closed
–
Safety: Critical or not
–
Sign off
HRA 2013 Annual Regional INCOSE Conference :
Systems Engineering and Systems Thinking – Back to Basics
Copyright © 2013 by Kevin Vipavetz LaRC, NASA
Permission granted to INCOSE to publish and use.
18
Verification Closure
Level: FTV System
Applicable documents: Ares I-X Control Algorithm Parameters : AI1-SYS-CAP, Ares-I-X Guidance,
Navigation, and Control System Design Document
Nonconformance history: None
Closure data/documentation required:
Verification Report: Ares I-X Similitude Trajectory: AI1-TAR-FTVTRAJ
Event preceding Verification Activity: Final Ares I-X GN&C Algorithms released, Final version of
Aerodynamics Loads, Final 6 DOF model of integrated vehicle
Estimated Duration of Verification Activity: _30__ Days
Closure Of FTV SRD Requirement
Date Closed:
Test engineer:
Requirement
owner:
[ N ] Safety Critical (Y/N)
Element/SE&I
S&MA:
Manager:
HRA 2013 Annual Regional INCOSE Conference :
Systems Engineering and Systems Thinking – Back to Basics
Copyright © 2013 by Kevin Vipavetz LaRC, NASA
Permission granted to INCOSE to publish and use.
19
Compliance Statement
• Requirement owner writes up a compliance statement as part of the
official closure of the requirements and is attached to the VRDS
• The compliance statement will show that all other linked verifications
at the lower levels have also been closed
• When the compliance statement is complete the signature process
can begin
HRA 2013 Annual Regional INCOSE Conference :
Systems Engineering and Systems Thinking – Back to Basics
Copyright © 2013 by Kevin Vipavetz LaRC, NASA
Permission granted to INCOSE to publish and use.
20
Compliance Statement Example, attaches to VRDS
VR-FTV-014 Compliance Statement: Review of the analysis conducted and
reported in the AI1-SYS-CAPv3.00 shows that the FTV is in compliance with SRD
requirement number FTV-014.
FTV-014: True Heading
 FTV-014: The FTV shall maintain a 90 +/- 2 degrees true heading angle from pitch over
to initiation of separation
[Rationale: Given that the Ares I-X mission exercises the FS burn only, achieving
approximately Mach 4.5, there are no orbital parameter requirements. Thus, the FTV
will follow a “due East” profile over the Atlantic Ocean upon launch. This sets up FS
recovery that meets P3 Test Objective.]
Comparisons of nominal and dispersed heading (psi) have been made with all the analysis tools
and are documented in the AI1-SYS-CAPv3, Chapter 9. Figure 1 (from page 169) shows the
nominal heading errors throughout the flight stay less than 1 degree for all analysis tools.
Figure 1 Heading, degrees off target azimuth.
HRA 2013 Annual Regional INCOSE Conference :
Systems Engineering and Systems Thinking – Back to Basics
Copyright © 2013 by Kevin Vipavetz LaRC, NASA
Permission granted to INCOSE to publish and use.
21
Compliance Statement Example cont…
Trace analysis
The following children requirements and artifacts traced to FTV-014:
AVI-017(Control Algorithm [Baseline])
CR-AIX-0588, Approved at XCB 20091006
FS-005(Thrust Vector Control)
FS-005-ELEC_ARES I-X-VSS-003 , Approved at XCB 20091006
FS-005-INT(STAGE)_ARES I-X-VSS-013 , Approved at XCB 20091010
Review of Artifacts and Closure reports
1. AIX-SYS-CAP
https://ice.exploration.nasa.gov/Windchill/netmarkets/jsp/document/download.jsp?oid=document%7Ewt.doc.WTDocument%3A
985878235&u8=1
2. Peer Review Documentation of AI1-SYS-CAP
https://ice.exploration.nasa.gov/Windchill/netmarkets/jsp/folder/view.jsp?oid=folder%7Ewt.folder.SubFolder%3A1616820362&
u8=1
3. VRDS-AVI-017
https://ice.exploration.nasa.gov/Windchill/netmarkets/jsp/folder/view.jsp?oid=folder%7Ewt.folder.SubFolder%3A1485222456&
u8=1
4. VRDS-FS-005
https://ice.exploration.nasa.gov/Windchill/netmarkets/jsp/document/download.jsp?oid=document%7Ewt.doc.WTDocument%3A
985878235&u8=1
HRA 2013 Annual Regional INCOSE Conference :
Systems Engineering and Systems Thinking – Back to Basics
Copyright © 2013 by Kevin Vipavetz LaRC, NASA
Permission granted to INCOSE to publish and use.
22
Configuration Control
• All requirements and verifications and their artifacts need to be under
configuration control
• This protects the information from being corrupted or changed accidentally
• This is for baseline documents and project deemed configuration items only
• Any changes under configuration control will need to go through the change
request process for approval
• Tip: Use your requirements management (RM) tool to keep your
configuration control items and generate your documents for review
• Tip: Make the approved changes in the RM tool
HRA 2013 Annual Regional INCOSE Conference :
Systems Engineering and Systems Thinking – Back to Basics
Copyright © 2013 by Kevin Vipavetz LaRC, NASA
Permission granted to INCOSE to publish and use.
23
One Complete Requirement Verification Example
HRA 2013 Annual Regional INCOSE Conference :
Systems Engineering and Systems Thinking – Back to Basics
Copyright © 2013 by Kevin Vipavetz LaRC, NASA
Permission granted to INCOSE to publish and use.
24
Summary
• Good requirements lead to good verifications
• Assign requirement owners
• Use a good Requirements Management Tool to manage your
requirements
• Follow the LMS-CP 5526 Requirements Management
guidelines
HRA 2013 Annual Regional INCOSE Conference :
Systems Engineering and Systems Thinking – Back to Basics
Copyright © 2013 by Kevin Vipavetz LaRC, NASA
Permission granted to INCOSE to publish and use.
25
Questions?
HRA 2013 Annual Regional INCOSE Conference :
Systems Engineering and Systems Thinking – Back to Basics
Copyright © 2013 by Kevin Vipavetz LaRC, NASA
Permission granted to INCOSE to publish and use.
26