1. No category

A Decision Analysis Methodology to Assess Customer
Value of BMD Weapon Systems
Alex Hernandez
05/07/15
Agenda
 Background

Modeling BMD Customer Value






2
My Approach:Value-Focused Thinking
Qualitative Value Model
Solution Design
Quantitative Value Model
Value Assessment
Applications
Background
Dawn of the Missile Age1
During World War II, the Unites States Army recognized the need to
develop a capability to defend against a weapon like the German V-2
Great Britain was able to destroy some of the V-2s
Although not very effective, this concept represented an innovative
approach to ballistic missile defense (BMD)



MDA
4
Ballistic Missile Proliferation - 1972
MDA
5
Ballistic Missile Proliferation - 2004
MDA
6
Why is missile proliferation a problem?



Ballistic missile technology is available on a wider scale to hostile countries
Threats are becoming more mobile, reliable, accurate, and longer range
Missile arsenal in hostile countries is growing every year
MDA
7
Decision Analysis Importance

The United States has defense cooperative programs with
allies that are interested in buying BMD systems

US Defense Contractors face challenges in identifying the
value proposition of BMD systems for their customers

Customer questions that arise2:





8
Which alternatives provide validated capabilities?
Can the alternatives be supported?
What are the risks?
What are the life-cycle costs for each alternative?
How do the alternatives compare to one another?
Decision Analysis Challenges
Provide BMD decision-makers with an objective,
traceable, and robust methodology to define system
value3
Methodology should take inputs that are restricted to
information actually available to the real decision makers


Methodology should use decision maker, operator, and
developer knowledge and judgement to define qualitative
and quantitative value measures

There are methods developed, but few applied to BMD!
9
Research Objectives

Formulate a methodology that qualitatively and
quantitatively models customer value of BMD weapon
systems

Guide BMD contractors in formulating a value
proposition

Provide competitive information that contractors may
use to fill weapon system performance gaps
10
Modeling BMD Customer Value
DOD Acquisition System
JCIDS – Joint Capabilities
Integration Development System
PPBE – Planning, Program, Budget,
and Execution
ICD – Initial Capabilities
Document
CDD – Capabilities Development
Document
12
http://acqnotes.com/acquisitions-main
DOD Acquisition System
AoA
Capability Requirements
(ICD, CDD)
JCIDS – Joint Capabilities
Integration Development System
PPBE – Planning, Program, Budget,
and Execution
ICD – Initial Capabilities
Document
CDD – Capabilities Development
Document
AoA – Analysis of Alternatives
13
http://acqnotes.com/acquisitions-main
Analysis of Alternatives (AoA) Process2
MoE – Measure of Effectiveness
MoP – Measure of Performance
TDD – Technical Description Document
14
My Approach: Value-Focused Thinking4
Alternative-Focused Thinking
• Identify Alternatives
• Specify Values
• Evaluate Alternatives

Advantages



15
• Specify Values
• Create Alternatives
• Evaluate Alternatives

Focuses directly on
alternatives of interest
Disadvantages

Value-Focused Thinking
Initial alternatives may not
meet customers intent
No process to iterate on
alternatives
Advantages



Generate better alternatives
Support strategic objectives
Disadvantages

Requires more resources
(time, cost, team members)
Alternative-Focused Thinking5
16
Value-Focused Thinking
Alternative-Focused Thinking5
17
Value-Focused Thinking
My Decision Analysis Methodology
• Problem Definition
• Quality Function
Deployment
• Idea Generation
• Alternative
Generation
• Alternative
Enhancement
• Capability Gaps
• Cost-Benefit
• Risk Assessment
• Recommendations
• Effectiveness
Analysis
• Cost Analysis
• Risk and
Uncertainty
Analysis
18
Note: Based on Gregory Parnell Systems Decision Process6
Comparison of AoA to My Methodology
Solution Design
Qualitative Value Model
MoE – Measure of Effectiveness
MoP – Measure of Performance
TDD – Technical Description Document
Quantitative Value Model
19
Qualitative Value Model
Value Hierarchy
Fundamental
Objective
Functional
Requirement
Objective
21
Objective
Functional
Requirement
Objective
Objective
Measure of
Effectiveness
Measure of
Effectiveness
Measure of
Effectiveness
Measure of
Effectiveness
Measure of
Effectiveness
Measure of
Effectiveness
Measure of
Effectiveness
Measure of
Effectiveness
Value Hierarchy
Fundamental
Objective
PROBLEM DEFINITION
Functional
Requirement
Objective
Objective
Functional
Requirement
Objective
Objective
QUALITY FUNCTION
Measure of DEPLOYMENT
Measure of
Measure of
Measure of
22
Effectiveness
Effectiveness
Effectiveness
Effectiveness
Measure of
Effectiveness
Measure of
Effectiveness
Measure of
Effectiveness
Measure of
Effectiveness
Problem Definition

Stakeholder Analysis

Wants, needs, desires, constraints, complaints




Fundamental Objective(s)

23
Interviews
Questionnaires
Surveys
Statement(s) of the most basic reason(s) for the decision
Quality Function Deployment7 (QFD)

Excellent tool to develop and prioritize functional
requirements, objectives, and measures of effectiveness

Ensures that the “voice of the customer” is reflected in
the ultimate decision

QFD provides a method for individuals involved at
various steps of the development process to
communicate with each other

24
efficiently translates the language of one tier in the value
hierarchy to the next via a family of houses
QFD Techniques7
Functional
Requirements
Main QFD Tools
- Affinity Diagram
- Tree Diagram
25
Objectives
Functional
Requirements
Customer
Desires
Target
Values

Measures of
Effectiveness
Objectives
Target
Values
Target
Values
- Matrix Diagram
- Prioritization Matrix
Solution Design
Idea Generation8

Mental Blocks


Perceptual, emotional, cultural, environmental, intellectual, and
expressive
Common Causes:






Overcome by:



27
Defining the problem too narrowly
Assuming there is only one right answer
Getting “hooked” in the first solution
Trying to get by with a solution that almost works (but really doesn’t)
Being distracted by irrelevant information
Practicing ways to increase creative ability
Developing a more positive attitude toward risk taking
Recognizing that “failure” is simply an event on the learning curve
unless you do not learn from the event – then it is a failure
Alternative Generation6

General Morphological Analysis (GMA)



28
Explores all the possible solutions to a complex problem
Process:
 Concisely formulate the problem to be solved
 Localize all parameters that might be important for the solution
 Construct a multidimensional matrix containing all possible solutions
 Assess all solutions against the purposes to be achieved
 Select suitable solutions for application or iterative morphological
study
Eliminate ideas that are logically, empirically, and normatively inconsistent
Alternative Enhancement6
Reduce the number
of alternatives that
will be considered


29
Needs Filter
Feasible Alternatives
Wants Filter
Desires Filter
Enhanced Alternatives
Quality

Needs – essential
criteria that must
exist
Wants – additional
features that
significantly enhance
alternative
Desires – features
that provide a margin
of excellence
Ideas
Number of Alternatives

Quantitative Value Model
Operational Effectiveness Analysis

Goal is to determine the value of alternatives in
performing functions

Ability to perform function is determined from:
estimates of the performance of each alternative
with respect to multiple measures of effectiveness

31
BMD Examples: defended area, probability of kill, time
of engagement, battlespace
Mathematical Value Equation

Additive Value Equation9
where,
v(x) is the alternative’s value
i = 1 to n is the number of the value measure
xi is the alternative’s score on the ith value measure
vi(xi) is the single dimensional value of a score of xi
wi is the weight of the ith value measure
32
Stacked Bar Graphs9


The “stacks” show the contribution of each measure
of effectiveness (attribute) used to compare
alternatives. Highest score wins.
Value gaps are the difference between the
alternative’s stacks and the ideal stack

Provides insight to the areas that can be improved to satisfy
customer values
Ideal
Alternative 1
Alternative 2
Alternative 3
33
MoE A
MoE B
MoE C
Cost, Risk, and Uncertainty
Life-Cycle Cost9


Total LCC combined
with effectiveness
analysis identifies
alternative(s) that
represent the best
joint value
LCC is very difficult
to derive, so Average
Unit Procurement
Cost (AUPC) is
frequently used
Value
Value vs. AUPC
AUPC ($ Millions)
34

Operational Risk5



Methodology for
assigning a level of
likelihood to the
occurrence of each
adverse event
Failure Mode and Effects
Analysis (FMEA)
Risk Priority Number
RPN = (P)(S)(D)
where,
P = probability of the failure
event occurring
S = severity of the event
D = probability that the event
would not be detected before
the user is aware of it
Effectiveness
Uncertainty9

Additive value model
allows for three sources
of uncertainty
 Alternative scores
 Value functions
 Weights
Sensitivity Plot
Value

Ideal
Alternative 1
Alternative 2
Alternative 3
MoE B Weighting %
Value Assessment
Value Assessment

Bring all qualitative and quantitative information
together and interpret its meaning through comparative
analysis

Comparing alternatives means the simultaneous
consideration of alternatives in terms of:

Effectiveness

Cost
Risk


Recommendations are made based on how well
alternatives’ meet customer values and priorities
36
Applications
Decision Analysis Applications








A Methodology to Assess the Utility of Future Space Systems – 1994
Multi-objective Decision Analysis of Theater Missile Defense
Architectures – 2000
Multi-attribute Utility Theory Applied on Analysis of Alternative for
500kv Transmission Lines Building – 2000
A Decision Support Tool for Construction Bidding – 2002
Utility Theory Model for Equipment Selection – 2005
Multi-attribute Utility Theory for Irrigation System Evaluation – 2006
A Multi-attribute Utility Theory-Based Method for Product Line
Selection – 2007
Parametric Multi-Attribute Utility Functions for Optimal Profit Under
Risk Constraints – 2011
38
Decision Analysis Applications








A Methodology to Assess the Utility of Future Space Systems – 1994
Multi-objective Decision Analysis of Theater Missile Defense
Architectures – 2000
Multi-attribute Utility Theory Applied on Analysis of Alternative for
500kv Transmission Lines Building – 2000
A Decision Support Tool for Construction Bidding – 2002
Utility Theory Model for Equipment Selection – 2005
Multi-attribute Utility Theory for Irrigation System Evaluation – 2006
A Multi-attribute Utility Theory-Based Method for Product Line
Selection – 2007
Parametric Multi-Attribute Utility Functions for Optimal Profit Under
Risk Constraints – 2011
39
Most Relevant Application


Gregory Parnell applied Alternative-Focused Thinking and
Multi-Objective Decision Analysis (MODA) to maximize
the value of a Theater Missile Defense (TMD)
architecture subject to life cycle budgetary constraints3
His research does not include:





40
Additional Weapon Systems (Aegis, Aegis Ashore, THAAD ER,
Arrow, etc.)
Off-board Sensors (Space and Ground Radar Systems)
Stakeholder Analysis
Sufficient Measures of Effectiveness (only PK)
Identification of Value Gaps – Chances to improve the
alternatives
Deliverables

Published thesis report that includes:




Explanation of the theory behind the methodology
Explanation of steps of the methodology for BMD
Methodology challenges unique to BMD
Unpublished Lockheed Martin report that includes:





41
Effectiveness Analysis Results
Life-Cycle Cost Analysis Results
Risk Analysis Results
Alternative Comparison
Capability/Value Gap Analysis
42
References
Dabrowski, John R. Missile Defense: The First Seventy Years. MDA, 8 Aug. 2013. PDF.
Anonymous, 2008. Analysis of Alternatives Handbook: A Practical Guide to Analyses of
Alternatives.PDF. Kirtland, NM.
Parnell, G. S., R. E. Metzger, J. Merrick, and R. Eilers. 2000. Multiobjective Decision Analysis of
Theater Missile Defense Architectures.PDF.
Keeney, R. L. 1992. Value-Focused Thinking: A Path to Creative Decisionmaking. Cambridge, MA:
Harvard University Press.
Blanchard, B. S. and Fabrycky, W.J. 2006. Systems Engineering and Analysis. Upper Saddle River,
NJ: Prentice Hall.
Parnell, G. S. 2008. Decision Making in Systems Engineering and Management. Hoboken, NJ:
Wiley.
Cohen, L. 1995. Quality Function Deployment: How to Make QFD Work for You. University of
Michigan: Addison-Wesley.
Fogler, S.H., LeBlanc, S.E., and Rizzo, B. 2013. Strategies for Creative Problem Solving. Upper
Saddle River, NJ: Prentice Hall
Parnell, G. S., Chapter 19, Value-Focused Thinking Using Multiple Objective Decision Analysis,
Methods for Conducting Military Operational Analysis: Best Practices in Use Throughout the
Department of Defense, Military Operations Research Society, Editors Andrew Loerch and
Larry Rainey, 2007.
1.
2.
3.
4.
5.
6.
7.
8.
9.
43