Developing and Implementing Verification Requirement Definitions, the “How to” of Verifying Requirements
Developing and Implementing Verification Requirement Definitions, the “How to” of Verifying Requirements Kevin Vipavetz Senior Systems Engineer Langley Research Center Email: [email protected] Phone 757-864-3817 HRA 2013 Annual Regional INCOSE Conference : Systems Engineering and Systems Thinking – Back to Basics Copyright © 2013 by Kevin Vipavetz LaRC, NASA Permission granted to INCOSE to publish and use. Agenda • • • • • • • • • Success Factors Requirement Owners Requirements Flow Verification Matrix Verification Sequence Developing your Verification Requirement Definition Sheet Compliance Statement Configuration Control Summary HRA 2013 Annual Regional INCOSE Conference : Systems Engineering and Systems Thinking – Back to Basics Copyright © 2013 by Kevin Vipavetz LaRC, NASA Permission granted to INCOSE to publish and use. 2 Key Verification Success Factors • Implementing system engineering best practices • Using common/consistent formats and processes for requirement and verification development • Start with good requirements • Having accountability (ownership). • Use a good requirements management tool • Good configuration control of requirements and all verification artifacts HRA 2013 Annual Regional INCOSE Conference : Systems Engineering and Systems Thinking – Back to Basics Copyright © 2013 by Kevin Vipavetz LaRC, NASA Permission granted to INCOSE to publish and use. 3 Independent Reviews Positive Lesson – Independent reviews contributed extensively to the success of the mission • Independent reviews provided a special oversight and help ensure verification processes are followed • Independent reviews provided an additional check that requirements are complete, well communicated, and reduce the possibility of missing requirements • Independent reviews helped enforce issues to closure • Independent reviews are interested in your success and provide in-depth detailed reports in areas you may not have considered • Recommend using independent reviews extensively for requirements and verification development HRA 2013 Annual Regional INCOSE Conference : Systems Engineering and Systems Thinking – Back to Basics Copyright © 2013 by Kevin Vipavetz LaRC, NASA Permission granted to INCOSE to publish and use. 4 Good Systems Engineering • One source is the NPR 7123.1B “NASA Systems Engineering Processes and Requirements” and is a good outline for Systems Engineering Management Plan (SEMP) • The SEMP determines how you will conduct your engineering processes for the project – Every project should have one – Outlines Verification Plan content • Also, an excellent source for good requirements management is Langley’s LMS-CP5526 “Product Requirements Development and Management” You can email me if you would like a copy of these: [email protected] HRA 2013 Annual Regional INCOSE Conference : Systems Engineering and Systems Thinking – Back to Basics Copyright © 2013 by Kevin Vipavetz LaRC, NASA Permission granted to INCOSE to publish and use. 5 Good Requirements • Good requirements are critical to having successful verifications • Use skilled requirement writers • Requirements should be: – – – – – – – – Product oriented Concise, using the “Who shall do What “ format Single statement Measureable Feasible Verifiable Contain rationales (key to defining good verification activities and success criteria) Traceable HRA 2013 Annual Regional INCOSE Conference : Systems Engineering and Systems Thinking – Back to Basics Copyright © 2013 by Kevin Vipavetz LaRC, NASA Permission granted to INCOSE to publish and use. 6 Requirement Owners • Requirement owners are key to getting the good requirements developed and verified – They are not the requirement manager – They are part of the whole project life cycle – • They are accountable for developing, verifying, and implementing the requirements o Accountability is not the same as responsibility o All are responsible, but only one is accountable Requirement owners will define the verification activities, success criteria, develop compliance reports, and start the signature process for verification approval HRA 2013 Annual Regional INCOSE Conference : Systems Engineering and Systems Thinking – Back to Basics Copyright © 2013 by Kevin Vipavetz LaRC, NASA Permission granted to INCOSE to publish and use. 7 Requirement Owners cont… • There should be a requirement owner assigned to all requirements • For each requirement there should be a corresponding verification requirement Note: Spend some time on how you want to resource load your requirement owners by how you want to verify your requirements. Distribute the work load. HRA 2013 Annual Regional INCOSE Conference : Systems Engineering and Systems Thinking – Back to Basics Copyright © 2013 by Kevin Vipavetz LaRC, NASA Permission granted to INCOSE to publish and use. 8 Requirements Flow and Levels Requirement Flow Ares I-X Flight Test Plan (Primary and Secondary Objectives) Ares I-X Flight Test Vehicle System Requirements L1 Verification Flow L2 First Stage Element Requirements Upper Stage Element Requirements Roll Control Element Requirements Crew Module/Launch Abort System Element Requirements Avionics Element Requirements First Stage End Item Spec Upper Stage End Item Spec Roll Control End Item Spec Crew Module/Launch Abort System End Item Spec Avionics End Item Spec HRA 2013 Annual Regional INCOSE Conference : Systems Engineering and Systems Thinking – Back to Basics Copyright © 2013 by Kevin Vipavetz LaRC, NASA Permission granted to INCOSE to publish and use. L3 L4 9 Requirement Flow Example FTP FTV SRD FS ERD Propulsion End Item Spec • Starting with: Flight Test Plan Primary and Secondary Objectives P1-P5, S1-S7 • Specific Flight Test Plan Constraint 5.1f states, “ To minimize development costs, booster avionics components should be primarily Space Shuttle RSRM heritage whenever possible.” •FTV – 097: The FTV shall use the Space Shuttle Solid Rocket Booster (SRB) heritage Thrust Vector Control (TVC) System. •[Trace: P1-P5, S1-S7, FTP Constraint 5.1f] •[Allocation: FS, Avionics] •FS – 005: The First Stage shall utilize the SRB heritage TVC subsystem designed in accordance with specification 10CEI-0001. •[Trace: FTV - 097] •[Allocation: Propulsion] • Specification 10CEI-0001 “ Part I Integrated Solid Rocket Booster for Operational Flights” • [Trace: FS -005] • [Allocation: N/A (reached the bottom of the product hierarchy)] HRA 2013 Annual Regional INCOSE Conference : Systems Engineering and Systems Thinking – Back to Basics Copyright © 2013 by Kevin Vipavetz LaRC, NASA Permission granted to INCOSE to publish and use. 10 Verification Compliance Matrix (VCM) • VCM is useful for tracking and reporting verification status for reviews Verification Verification Verification Meets Success Requirement Number Method Criteria Title VR-FTV014 True Heading Analysis Yes Closure Artifact No./Title Result Summary Applicable Documents Models show the Ares I-X nominal heading AI1-SYS-CAP, Ares-I-X Similitude errors throughout Guidance, Navigation, Trajectory: AI1- the flight stay less and Control System TAR-FTVTRAJ than 1 degree for Design Document all analysis tools. HRA 2013 Annual Regional INCOSE Conference : Systems Engineering and Systems Thinking – Back to Basics NonRequirement Owner conformance None Vipavetz Copyright © 2013 by Kevin Vipavetz LaRC, NASA Permission granted to INCOSE to publish and use. 11 Verification Sequence HRA 2013 Annual Regional INCOSE Conference : Systems Engineering and Systems Thinking – Back to Basics Copyright © 2013 by Kevin Vipavetz LaRC, NASA Permission granted to INCOSE to publish and use. 12 Requirement Example 1.1.1.1 FTV-014: True Heading [Baseline] The FTV shall maintain a 90 +/- 2 degrees true heading angle from pitch over to initiation of separation [Rationale: Given that the Ares I-X mission exercises the first stage burn only, achieving approximately Mach 4.5, there are no orbital parameter requirements. Thus, the FTV will need to follow a “due East” profile over the Atlantic Ocean upon launch. This sets up FS recovery that meets P3 Test Objective.] [Trace: FTP Constraint 5.1e, P1, P3] [Requirement Owner] Kevin Vipavetz (Vip) [Verification Method] Analysis [Allocation: FS, Avionics] [Priority: 2] HRA 2013 Annual Regional INCOSE Conference : Systems Engineering and Systems Thinking – Back to Basics Copyright © 2013 by Kevin Vipavetz LaRC, NASA Permission granted to INCOSE to publish and use. 13 Developing Verification Activity and Success Criteria • • • • The requirement owner is the one who fills out the Verification Definition Sheets and work the activities and criteria The requirement rationale and performance or function of the requirement will help determine what your verification activity and success criteria will be If you are having trouble with this part of the verification then most likely you have a poor requirement written VR-FTV-014 activity will use a comparison of models for analysis and the success criteria is the heading must stay under +/- 2 degrees for all the models HRA 2013 Annual Regional INCOSE Conference : Systems Engineering and Systems Thinking – Back to Basics Copyright © 2013 by Kevin Vipavetz LaRC, NASA Permission granted to INCOSE to publish and use. 14 Verification Requirement Definition Sheet (VRDS) • Defines how you do the verification process for each requirement • It contains the verification implementation and closeout information HRA 2013 Annual Regional INCOSE Conference : Systems Engineering and Systems Thinking – Back to Basics Copyright © 2013 by Kevin Vipavetz LaRC, NASA Permission granted to INCOSE to publish and use. 15 VRDS Terms Top Portion • Verification Implementation section – Verification number: Unique ID linked to corresponding requirement – Requirement Title: Name of you verification requirement – Verification Method: Inspection, Demonstration, Analysis or Test – Verification Activity: Description of how the verification is carried out – Success Criteria: What was the criteria used to know we are compliant – Rationale: Why the this verification method was chosen HRA 2013 Annual Regional INCOSE Conference : Systems Engineering and Systems Thinking – Back to Basics Copyright © 2013 by Kevin Vipavetz LaRC, NASA Permission granted to INCOSE to publish and use. 16 Ares I–X FTV System Verification Definition Sheet FTV SRD Requirement to be Verified Verification Number: Requirement Number: Priority (1-3): VR-FTV-014 FTV-014 2 Requirement Title: True Heading Verification Implementation Verification method: The maintenance of the true heading angle defined in the AI1-SYS-SRD shall be verified by Analysis. Description of verification activities to be performed: The analysis shall be done using a NASA approved 6-DOF non-linear flight dynamics model of the integrated vehicle. The analysis shall be done using Monte Carlo techniques with an ensemble of input values that model up to 3-sigma dispersions for vehicle performance and environmental conditions. Success Criterion: The verification shall be considered successful when the analysis shows that the true heading angle is less than +/- 2 degrees during flight (from pitch over to initiation of separation) with a probability of at least 95%. Rationale: It is not possible to adequately test the flight dynamical behavior of the FTV in a ground-based environment prior to launch, thus analysis must be used. HRA 2013 Annual Regional INCOSE Conference : Systems Engineering and Systems Thinking – Back to Basics Copyright © 2013 by Kevin Vipavetz LaRC, NASA Permission granted to INCOSE to publish and use. 17 VRDS Terms Bottom Portion • Verification Closeout – Non-Conformance reports: Deviations, waivers, or failures – Closure Report number: Unique ID tracking number – Applicable Documents: Procedures, drawings, manuals etc. used for the verification – Verification Report: Final closeout report – Event preceding verification activity: Used to assist scheduling – Estimated Duration of Verification Activity: Used to assist scheduling – Verification Closure Date: Date closed – Safety: Critical or not – Sign off HRA 2013 Annual Regional INCOSE Conference : Systems Engineering and Systems Thinking – Back to Basics Copyright © 2013 by Kevin Vipavetz LaRC, NASA Permission granted to INCOSE to publish and use. 18 Verification Closure Level: FTV System Applicable documents: Ares I-X Control Algorithm Parameters : AI1-SYS-CAP, Ares-I-X Guidance, Navigation, and Control System Design Document Nonconformance history: None Closure data/documentation required: Verification Report: Ares I-X Similitude Trajectory: AI1-TAR-FTVTRAJ Event preceding Verification Activity: Final Ares I-X GN&C Algorithms released, Final version of Aerodynamics Loads, Final 6 DOF model of integrated vehicle Estimated Duration of Verification Activity: _30__ Days Closure Of FTV SRD Requirement Date Closed: Test engineer: Requirement owner: [ N ] Safety Critical (Y/N) Element/SE&I S&MA: Manager: HRA 2013 Annual Regional INCOSE Conference : Systems Engineering and Systems Thinking – Back to Basics Copyright © 2013 by Kevin Vipavetz LaRC, NASA Permission granted to INCOSE to publish and use. 19 Compliance Statement • Requirement owner writes up a compliance statement as part of the official closure of the requirements and is attached to the VRDS • The compliance statement will show that all other linked verifications at the lower levels have also been closed • When the compliance statement is complete the signature process can begin HRA 2013 Annual Regional INCOSE Conference : Systems Engineering and Systems Thinking – Back to Basics Copyright © 2013 by Kevin Vipavetz LaRC, NASA Permission granted to INCOSE to publish and use. 20 Compliance Statement Example, attaches to VRDS VR-FTV-014 Compliance Statement: Review of the analysis conducted and reported in the AI1-SYS-CAPv3.00 shows that the FTV is in compliance with SRD requirement number FTV-014. FTV-014: True Heading FTV-014: The FTV shall maintain a 90 +/- 2 degrees true heading angle from pitch over to initiation of separation [Rationale: Given that the Ares I-X mission exercises the FS burn only, achieving approximately Mach 4.5, there are no orbital parameter requirements. Thus, the FTV will follow a “due East” profile over the Atlantic Ocean upon launch. This sets up FS recovery that meets P3 Test Objective.] Comparisons of nominal and dispersed heading (psi) have been made with all the analysis tools and are documented in the AI1-SYS-CAPv3, Chapter 9. Figure 1 (from page 169) shows the nominal heading errors throughout the flight stay less than 1 degree for all analysis tools. Figure 1 Heading, degrees off target azimuth. HRA 2013 Annual Regional INCOSE Conference : Systems Engineering and Systems Thinking – Back to Basics Copyright © 2013 by Kevin Vipavetz LaRC, NASA Permission granted to INCOSE to publish and use. 21 Compliance Statement Example cont… Trace analysis The following children requirements and artifacts traced to FTV-014: AVI-017(Control Algorithm [Baseline]) CR-AIX-0588, Approved at XCB 20091006 FS-005(Thrust Vector Control) FS-005-ELEC_ARES I-X-VSS-003 , Approved at XCB 20091006 FS-005-INT(STAGE)_ARES I-X-VSS-013 , Approved at XCB 20091010 Review of Artifacts and Closure reports 1. AIX-SYS-CAP https://ice.exploration.nasa.gov/Windchill/netmarkets/jsp/document/download.jsp?oid=document%7Ewt.doc.WTDocument%3A 985878235&u8=1 2. Peer Review Documentation of AI1-SYS-CAP https://ice.exploration.nasa.gov/Windchill/netmarkets/jsp/folder/view.jsp?oid=folder%7Ewt.folder.SubFolder%3A1616820362& u8=1 3. VRDS-AVI-017 https://ice.exploration.nasa.gov/Windchill/netmarkets/jsp/folder/view.jsp?oid=folder%7Ewt.folder.SubFolder%3A1485222456& u8=1 4. VRDS-FS-005 https://ice.exploration.nasa.gov/Windchill/netmarkets/jsp/document/download.jsp?oid=document%7Ewt.doc.WTDocument%3A 985878235&u8=1 HRA 2013 Annual Regional INCOSE Conference : Systems Engineering and Systems Thinking – Back to Basics Copyright © 2013 by Kevin Vipavetz LaRC, NASA Permission granted to INCOSE to publish and use. 22 Configuration Control • All requirements and verifications and their artifacts need to be under configuration control • This protects the information from being corrupted or changed accidentally • This is for baseline documents and project deemed configuration items only • Any changes under configuration control will need to go through the change request process for approval • Tip: Use your requirements management (RM) tool to keep your configuration control items and generate your documents for review • Tip: Make the approved changes in the RM tool HRA 2013 Annual Regional INCOSE Conference : Systems Engineering and Systems Thinking – Back to Basics Copyright © 2013 by Kevin Vipavetz LaRC, NASA Permission granted to INCOSE to publish and use. 23 One Complete Requirement Verification Example HRA 2013 Annual Regional INCOSE Conference : Systems Engineering and Systems Thinking – Back to Basics Copyright © 2013 by Kevin Vipavetz LaRC, NASA Permission granted to INCOSE to publish and use. 24 Summary • Good requirements lead to good verifications • Assign requirement owners • Use a good Requirements Management Tool to manage your requirements • Follow the LMS-CP 5526 Requirements Management guidelines HRA 2013 Annual Regional INCOSE Conference : Systems Engineering and Systems Thinking – Back to Basics Copyright © 2013 by Kevin Vipavetz LaRC, NASA Permission granted to INCOSE to publish and use. 25 Questions? HRA 2013 Annual Regional INCOSE Conference : Systems Engineering and Systems Thinking – Back to Basics Copyright © 2013 by Kevin Vipavetz LaRC, NASA Permission granted to INCOSE to publish and use. 26
© Copyright 2024