Outcome Monitoring Feedback
with the OQ-45:
Theory, Research and Practice
Kim de Jong, PhD
20-03-2015
Discover the world at Leiden University
Introductory exercise
Discover the world at Leiden University
Content
•
•
•
•
Part I
Why do we need feedback?
Description of the Outcome Questionnaire-45
Types of outcome measurement
Example cases
Part II
• Feedback theory
• Exercise
• Questions
Discover the world at Leiden University
Why do we need feedback?
I’m not that
satisfied with
the therapy…
Hmm… yes… I
understand…
That is part of
your psychiatric
problems.
Discover the world at Leiden University
“Without direct feedback on
how their clients are
progressing, clinicians are
essentially wearing a blindfold
while shooting at a target”
(Sapyta, Riemer & Bickman, 2005)
Discover the world at Leiden University
Therapists’ predictions of outcome
• Final outcome was predicted for 550 clients
• 3 were predicted to have a negative outcome,
whereas 40 had actual negative outcomes
• Staff were accurate 1 time (2,5%)
• Algorithms for feedback were correct 77% of
the time in predicting deteriorated patients
(Hannan, Lambert, Harmon, Nielsen, Smart, Shimokawa, Sutton, 2005)
Discover the world at Leiden University
Therapists’ predictions of outcome (2)
Expected recovery
Actual recovery
64,1
61,9
34,9
Control
n=169; De Jong & Peetoom, in preparation
32,8
Feedback
Discover the world at Leiden University
Do we overestimate ourselves?
• Self-serving bias:
• 2/3 of clinicians consider themselves to be in the
top 75% of their field
• 0% belief themselves to be below average
• Therapy rationale:
• Do patients need to get worse before they get
better?
Discover the world at Leiden University
The evidence on OM feedback (1)
• Feedback improves communication between patient
and clinician (Carlier et al., 2010)
• Feedback improves accuracy of diagnosis (Carlier et al.,
2010)
• Effects on outcome:
• Often no effect on “on track” cases
• Effect on “not on track” cases ranges from small to large
effect sizes (e.g. Bickman et al, 2012; d = 0.20) Shimokawa et al., 2010, d=0.92)
• Some studies show effect on treatment duration
Discover the world at Leiden University
The evidence on OM feedback (2)
• Largest effects found by Lambert e.a.:
• High frequency (every session)
• Expected recovery curves and warning signals (red
flag)
• Clinical support tools on common factors
Discover the world at Leiden University
Outcome Questionnaire
• Brief: 45 items, administration in 5 min
• Three domains:
• Symptomatic Distress
• Problems in Interpersonal Relations (partner,
family, friends)
• Social Role problems (work, free time)
• Translated into more than 20 languages
Discover the world at Leiden University
Outcome Questionnaire (2)
• Good psychometric properties (Amble et al., 2014):
•
•
•
•
α across language versions 0.91-0.96
test-retest reliability 0.84-0.91
Good concurrent validity
Sensitive to change
Discover the world at Leiden University
Symptom Distress
•
•
•
•
•
•
•
I feel no interest in things
I blame myself for things
I feel that something bad is going to happen
I am a happy person
I feel worthless
I feel lonely
Disturbing thoughts come into my mind that I
can’t get rid of
• I like myself
Discover the world at Leiden University
Interpersonal Relations
• I get along well with others
• I feel unhappy in my marriage/significant
relationship
• I am concerned about family troubles
• I have frequent arguments
• I have trouble getting along with friends and
close acquaintances
• I feel my love relationships are full and complete
Discover the world at Leiden University
Social Role
•
•
•
•
•
I feel stressed at work / school
I find my work/school satisfying
I work/study too much
I am working/studying less well than I used to
I have trouble at work/school because of
drinking or drugs use
• I feel that I am not doing well at work/school
• I have too many disagreements at work/school
Discover the world at Leiden University
Risk items
• Flagged when a patient endorses > never
• I have thoughts of ending my life
• After heavy drinking, I need a drink the next morning
to get going
• I feel annoyed by people who criticize my drinking or
drug use
• I have trouble at work/school because of drinking or
drug use
• I feel angry enough at work/school to do something I
might regret
Discover the world at Leiden University
Types of outcome measurement
• Pre-post: outcome measurement
• Multiple: outcome monitoring
• Continuous: outcome tracking
Additional options:
• Adding diagnosis specific instruments
• Use of expected recovery curves
• Use of clinical support tools
Discover the world at Leiden University
Pre-post measurement
70
60
50
Total
SD
IR
SR
40
30
20
10
0
Pre-treatment
Post-treatment
Discover the world at Leiden University
35
30
score
25
20
BDI
15
10
5
0
1
2
3
4
5
6
7
8
9
10 11 12 13
Discover the world at Leiden University
Outcome tracking
Discover the world at Leiden University
Discover the world at Leiden University
Expected recovery curves
Discover the world at Leiden University
Clinical support tools
Discover the world at Leiden University
Therapeutic relationship manual
• Discuss the agreement between you and your
client in treatment goals
• Explain your techniques, actions or behaviors
to the client
• Provide and ask for feedback on the working
alliance
• Accept responsibility for your share in the
problems
Discover the world at Leiden University
BREAK
Discover the world at Leiden University
Imagine that you get feedback that your patient
is not progressing well, whereas you thought
things were going ok….
... how does it make you feel?
… what do you do?
Discover the world at Leiden University
FEEDBACK THEORY
Discover the world at Leiden University
Feedback: the three elements
SOURCE
Perceived
validity
X
MESSAGE
Valence (+ / -)
X
RECEIVER
Feedback
propensity
Content
Self-efficacy
Timing
Discover the world at Leiden University
Feedback theory
Goal
commitment
Goal
or standard
Self-efficacy
Behavior
change
Valence (+ / -)
Content
Timing
Feedback
Locus of
control
Feedback
propensity
Perceived
validity
Adaptation of Concentualized Feedback Intervention Theory. Riemer & Bickman, 2011
Discover the world at Leiden University
Goal commitment
Goal
commitment
Expectations
Self-efficacy
Barriers
Attractiveness
Support
Higher goals
Discover the world at Leiden University
Attention and acceptance
• Validity of the source
• Usefulness
• Specificity
• Content
• Lay-out
• Feedback preference (propensity)
Discover the world at Leiden University
Causal attribution
Causality
Control
Stability
Discover the world at Leiden University
Exercise
• How would you motivate (fellow) therapists
to start using outcome monitoring feedback?
•
•
•
•
What motivates therapists /are higher goals?
What are promoting factors?
What are the barriers in your organization?
What can you control as implementer and what
not?
Discover the world at Leiden University
Effect feedback on outcome
95
OQ-45 Total score
85
75
NOT controle
NOT feedback
65
OT controle
55
OT feedback
45
35
Start
Eind
Discover the world at Leiden University
Effect feedback active use
95
OQ-45 Total score
85
75
NOT controle
NOT feedback
65
OT controle
55
OT feedback
45
35
Start
Eind
Discover the world at Leiden University
Therapist characteristics
• Effect on outcomes:
• Therapists with higher internal feedback propensity
achieve change slower
• Therapists with a positive attitude towards ROM , achieve
change faster
• Therapists with a higher self-efficacy achieve faster change
in NOT cases
(de Jong et al., 2012)
Discover the world at Leiden University
Therapist effects on use of feedback
Discover the world at Leiden University
QUESTIONS
www.kimdejong.net (English/Dutch mix)
Follow me on twitter: kdej_psyres
Discover the world at Leiden University