• Why is it so hard to get
organizations to use DA methods?
• Why do some managers accept DA
with open arms, but others resist?
• Why do some firms adopt DA
methods, but others do not?
• Are we doing something wrong?
1
Who uses Decision Analysis,
and Why?
Bob Clemen
Duke University
With Kelly E. See and Guy McCumber
Paper: See, K. E., & Clemen, R. T. (2006). Psychological and organizational factors
influencing decision process innovation: The role of perceived threat to managerial
power. Draft. Download from: http://faculty.fuqua.duke.edu/~clemen/bio/work.htm
1
Outline
• A survey of senior managers
– Whom did we survey?
– What did we ask?
• Results
– What did we learn?
• Wait until the end – bonus “short subject”
3
Remember back to 2000
• “Value of DA” – informal survey of 18
DAAG members
• We learned a little about
– How many, what kind of decisions DA
used for
– Little or no tracking of DA value added
– Ways DA adds non-monetary value
– Detractors of DA? (Yes)
4
2
Survey of Senior Managers
• What factors influence the use of decision
analysis (DA) in organizations?
• Individual factors
–
–
–
–
Attitude toward change?
Aversion to technology?
Threat to managerial control and value?
Perceived usefulness of DA?
• Organizational factors
–
–
–
–
Organizational culture of change?
Centralized decision making?
Lots of formal procedures?
Need to justify?
• Industry environment
5
Whom did we ask?
• 160 senior managers
(average experience = 17 years)
• All students or alumni of Duke’s Global
Executive MBA
• All exposed to decision analysis
• Responded to survey online
• Around 100 questions
6
3
Two Categories of DA
1. Qualitative problem-structuring methods (“soft DA”)
• Understanding objectives,
preferences (e.g., “Value-focused
thinking”)
• Creating new alternatives (e.g.,
Strategy tables)
• Strategy tables
• Decision hierarchy
• Influence diagrams
• Including stakeholders (e.g., Dialogue
decision process)
• Other group decision making
techniques
2. Quantitative and technical tools (“hard DA”)
• Scenario analysis
• Sensitivity analysis (e.g., Tornado
diagrams)
• Identifying ranges (best
case/worst case) of outcomes
• Probability distributions for
outcomes (risk profiles)
• Expert judgment
• Monte Carlo simulation
•
•
•
•
•
•
•
•
Decision trees
Value of information, control
Real options
Optimization methods
Risk tolerance
Cost-benefit analysis
Multiattribute utility
Decision support systems, “expert
systems”
• … and others
7
Basic Results
• High incidence of DA use: 70%
• Top qualitative methods:
– Create alternatives
– Understand objectives
– Org process (DDP)
61%
59%
53%
• Top quantitative methods
– Cost-benefit analysis
– Ranges of outcomes
– Scenario analysis
68%
63%
59%
8
4
Va
lu
eFo
c
Cr use
ea d
te Th
Al ink
St ter ing
na
r
a
D
ec teg tiv
es
is
y
In ion Ta
D
ia
lo flue Hi ble
gu
s
nc er
O e D e D arc
th
ia hy
er eci
gr
G si o
ro
n am
s
up Pr
Sc Te oce
en ch ss
ni
Se ar
qu
ns io
An e s
it
al
Id ivit
ys
en y
tif An is
a
yi
ng lys
Ra is
Pr ng
ob es
Ex Ris abi
li
pe k
Pr ty
rt
Ju ofi
dg les
M em
on
e
Va De te nt
Ca
lu
ci
si
e
o n rl o
Re of
al
In
Tr
f
e
O
pt o/C es
io
ns ont
An rol
O
pt aly
U
si
i
m
til
s
ity iza
Fu tio
nc n
Co
t
st ion
-B
s
M
en
ul
ef
tia Tr
a
i
t
tt
d
rib eut Of
fs
O
e
th
U
til
er
ity
Co
m
pu DS
S
te
r
Ai
ds
Basic Results
• DA used in 30% of decisions (median)
• 55% “outsource” (use consultants)
• 41% offer training resources
• 36% have standard operating
procedures for decision making
9
Tool Use By Industry
5.00
4.50
Soft DA
Manufacturing
Hard DA
4.00
3.50
3.00
2.50
2.00
1.50
1.00
0.50
0.00
Finance, Insurance, Banking, Real Estate
Services
10
5
lu
e
-F
oc
Cr use
ea d
te Th
Al ink
St ter ing
na
r
a
D
ec teg tiv
es
is
y
In ion Ta
D
ia
lo flue Hi ble
gu
s
nc e r
O e D e D arc
th
ia hy
er eci
gr
G si o
ro
n am
s
up Pr
Sc Te oce
ss
en ch
ni
S e ar
qu
ns io
An es
it
al
Id ivit
ys
en y
tif An is
a
yi
ng lys
Ra i s
Pr ng
ob es
Ex Ris abi
li
pe k
Pr ty
rt
Ju ofi
dg les
M em
on
e
Va De te nt
Ca
lu
ci
si
e
on rlo
Re
o
al f In
Tr
f
e
O
pt o/C es
io
ns ont
An rol
O
pt aly
U
si
i
m
til
s
ity iza
Fu tio
nc n
Co
t
st ion
-B
s
M
en
ul
ti a Tr
ef
a
i
t
tt
d
rib eut Of
fs
O
e
th
U
til
er
ity
Co
m
pu DS
S
te
r
Ai
ds
Va
lu
e
Cr
-F
oc
us
ea ed
te Th
Al ink
St ter ing
na
r
a
D
ec teg tiv
es
is
y
In ion Ta
D
ia
lo flue Hi ble
gu
s
nc e r
O e D e D arc
th
ia hy
er eci
gr
G si o
ro
n am
s
up Pr
Sc Te oce
ss
en ch
ni
S e ar
qu
ns io
An es
it
al
Id ivit
ys
en y
tif An is
a
yi
ng lys
Ra i s
Pr ng
ob es
Ex Ris abi
li
pe k
Pr ty
rt
Ju ofi
dg les
M em
on
e
Va De te nt
Ca
lu
ci
si
e
on rlo
Re
o
al f In
Tr
f
e
O
pt o/C es
io
ns ont
An rol
O
pt aly
U
si
i
m
til
s
ity iza
Fu tio
nc n
Co
t
st ion
-B
s
M
en
ul
ti a Tr
ef
a
i
t
tt
d
rib eut Of
fs
O
e
th
U
til
er
ity
Co
m
pu DS
S
te
r
Ai
ds
Va
Company Size & Average Tool Use
4.50
4.00
4.00
Soft DA
Hard DA
3.50
3.00
2.50
2.00
1.50
1.00
0.50
0.00
0-99 (N=23)
100-999 (N=31)
Soft DA
<$10M (N=35)
1000-9999 (N=26)
>10000 (N=23)
11
R&D Budget and Average Tool Use
4.50
Hard DA
3.50
3.00
2.50
2.00
1.50
1.00
0.50
0.00
>$10M (N=29)
12
6
lu
e
Cr
-F
oc
us
ea ed
te Th
Al ink
St ter ing
na
r
a
D
ec teg tiv
es
is
y
In ion Ta
D
ia
lo flue Hi ble
gu
s
nc e r
O e D e D arc
th
ia hy
er eci
gr
G si o
ro
n am
s
up Pr
Sc Te oce
ss
en ch
ni
S e ar
qu
ns io
An es
it
al
Id ivit
ys
en y
tif An is
a
yi
ng lys
Ra i s
Pr ng
ob es
Ex Ris abi
li
pe k
Pr ty
rt
Ju ofi
dg les
M em
on
e
Va De te nt
Ca
lu
ci
si
e
on rlo
Re
o
al f In
Tr
f
e
O
pt o/C es
io
ns ont
An rol
O
pt aly
U
si
i
m
til
s
ity iza
Fu tio
nc n
Co
t
st ion
-B
s
M
en
ul
ti a Tr
ef
a
i
t
tt
d
rib eut Of
fs
O
e
th
U
til
er
ity
Co
m
pu DS
S
te
r
Ai
ds
Va
Uncertainty (Mean Split of envunc2) and Average Tool Use
4.00
3.50
Soft DA
Hard DA
3.00
2.50
2.00
1.50
1.00
0.50
0.00
Low Uncertainty
High Uncertainty
13
Results from regression analysis:
Individual-level variables
Relationship
• Technical aversion
• Ease of use of DA tool
Demonstrate positive
results
• Diminishes managerial
value, discretion, control
negative
positive
negative
14
7
lu
e
-F
oc
Cr use
ea d
te Th
Al ink
St ter ing
na
r
a
D
ec teg tiv
es
is
y
In ion Ta
D
ia
lo flue Hi ble
gu
s
nc e r
O e D e D arc
th
ia hy
er eci
gr
G si o
ro
n am
s
up Pr
Sc Te oce
ss
en ch
ni
S e ar
qu
ns io
An es
it
al
Id ivit
ys
en y
tif An is
a
yi
ng lys
Ra i s
Pr ng
ob es
Ex Ris abi
li
pe k
Pr ty
rt
Ju ofi
dg les
M em
on
e
Va De te nt
Ca
lu
ci
si
e
on rlo
Re
o
al f In
Tr
f
e
O
pt o/C es
io
ns ont
An rol
O
pt aly
U
si
i
m
til
s
ity iza
Fu tio
nc n
Co
t
st ion
-B
s
M
en
ul
ti a Tr
ef
a
i
t
tt
d
rib eut Of
fs
O
e
th
U
til
er
ity
Co
m
pu DS
S
te
r
Ai
ds
Va
c
Cr use
d
ea
te Th
Al ink
St ter ing
na
D rat
tiv
e
ec
es
i s gy
In ion Ta
D
ia
f
bl
lu
H
lo
es
ie
gu en
r
c
O e D e D arc
th
h
ia
y
er eci
gr
G si o
ro
n am
s
up Pr
Sc Te oce
en ch ss
ni
Se ar
qu
ns io
An es
it
al
Id ivit
ys
en y
tif An is
a
yi
ng lys
Ra is
Pr ng
o b es
Ex Ris abi
li
pe k
Pr ty
rt
Ju ofi
dg les
M em
on
e
Va De te nt
Ca
lu
ci
si
e
o n rl o
Re
o
al f In
Tr
f
e
O
pt o/C es
io
ns ont
An rol
O
pt aly
U
si
i
m
til
s
ity iza
Fu tio
nc n
Co
t
st ion
-B
s
M
en
ul
t i a Tr
e
tt ade fit
rib
ut Of
fs
O
e
th
U
til
er
ity
Co
m
pu DS
S
te
r
Ai
ds
Va
lu
eFo
Organization-level variables
Organizational Culture of Change (Mean-Split of "org chang") and Average Tool Use
4.0000
3.5000
4.50
Soft DA
Hard DA
3.0000
2.5000
2.0000
1.5000
1.0000
0.5000
0.0000
Little Change
Soft DA
Low Centralization
Much Change
15
Centralization & Average Tool Use
5.00
Hard DA
4.00
3.50
3.00
2.50
2.00
1.50
1.00
0.50
0.00
High Centralization
16
8
Va
lu
eFo
c
Cr use
ea d
te Th
Al ink
St ter ing
na
r
a
D
ec teg tiv
es
is
y
In ion Ta
D
ia
lo flue Hi ble
gu
s
nc e r
O e D e D a rc
th
ia hy
er eci
gr
G sio
ro
n am
s
u p Pr
Sc Te oce
ss
en ch
ni
Se ar
qu
ns io
es
A
it
Id ivit nal
ys
en y
is
A
tif
yi nal
ng ys
Ra is
Pr n g
ob es
Ex Ris abi
li
pe k
Pr t y
rt
Ju ofi
dg les
M em
on
e
Va De te nt
Ca
lu
ci
si
e
on rlo
Re of
al
I
T
O nfo ree
pt
/
io Co s
nt
ns
An rol
O
pt aly
U
til imi sis
ity za
Fu tio
n
n
Co c t
st ion
s
B
M
e
ul
tia Tr nef
tt ade it
rib
ut Of
fs
O
e
th
U
til
er
ity
Co
m
pu DS
S
te
r
Ai
ds
lu
e
Cr
-F
oc
us
ea ed
te Th
Al ink
St ter ing
na
r
a
D
ec teg tiv
es
is
y
In ion Ta
D
ia
lo flue Hi ble
gu
s
nc e r
O e D e D arc
th
ia hy
er eci
gr
G si o
ro
n am
s
up Pr
Sc Te oce
ss
en ch
ni
S e ar
qu
ns io
An es
it
al
Id ivit
ys
en y
tif An is
a
yi
ng lys
Ra i s
Pr ng
ob es
Ex Ris abi
li
pe k
Pr ty
rt
Ju ofi
dg les
M em
on
e
Va De te nt
Ca
lu
ci
si
e
on rlo
Re
o
al f In
Tr
f
e
O
pt o/C es
io
ns ont
An rol
O
pt aly
U
si
i
m
til
s
ity iza
Fu tio
nc n
Co
t
st ion
-B
s
M
en
ul
ti a Tr
ef
a
i
t
tt
d
rib eut Of
fs
O
e
th
U
til
er
ity
Co
m
pu DS
S
te
r
Ai
ds
Va
Need for Legitimacy (Mean-Split of legit) and Average Tool Use
4.5000
4.0000
3.50
Soft DA
Hard DA
3.5000
3.0000
2.5000
2.0000
1.5000
1.0000
0.5000
0.0000
Low Need for Legitimacy
Soft DA
Low Formalization
High Need for Legitimacy
17
Formalization & Average Tool Use
4.00
Hard DA
3.00
2.50
2.00
1.50
1.00
0.50
0.00
High Formalization
18
9
But …
• Low formalization
– Perceived threat matters. More perceived
threat, less DA use
• High formalization
– Perceived threat doesn’t matter
• Why?
19
Implications
• Some variables make sense
– Individual: Tech aversion, ease of use, perceived
threat
– Organizational: Legitimacy, Org change culture
• Others may be less obvious
– Low centralization
– Formalization × perceived mgr threat
• Consider doing an organizational “audit”
– Highlight strengths and weaknesses of the
organization
– Develop implementation plan guided by audit
results
20
10
Bonus “Short Subject”
• Rex Brown’s recent Interfaces article, “The Operation
was a Success but the Patient Died.”
– Argues that outside consultant’s
priorities may differ from client’s
Æ Failure of DA project.
• We asked questions that can be used to address this:
– Does your organization outsource DA (use consultants)?
– In your experience, has DA been poorly executed?
• Some research questions:
CT
E
F
EF
– Outside consultants vs. in-house DA experts?
– What about extent of DA use, or how many different tools
used?
NO
Stay tuned! …
21
11