1. No category

Evidence-Based Medicine
How to appraise a
clinical trial
Hassan Gorji MD, MSc
Assistant Professor
NOSM
1
How much should we believe the results?
2
Objectives of workshop
• How a clinical trial works
• Appraise the main steps of a clinical trial
• Evaluate the applicability of trial finding to
specific patient or clinical setting
3
4
Formulate
your question
Don’t know
what to do
Design your search
strategy
Patient
Select related
studies
RCT
How good is it?
Should I trust the results?
5
Should I trust the
results?
6
Clinical scenario
You have a patient with peptic ulcer disease and stable angina.
He is on low dose aspirin, a statin, ACEI and as needed nitrate.
A week before he had an episode
of GI bleeding. Endoscopy was done. HP is negative.
You want to know is it better to change aspirin to
clopidogrel or just add a PPI to aspirin.
7
Formulating the question
Population
Patients
with peptic ulcer disease
and stable angina who had
a history of
GI bleeding
Intervention
Clopidogrel
Comparison
Aspirin and PPI
Outcome
Risk of GI bleeding
8
With The written Permission of
DR. Francis KL Chan
Department of Medicine and Therapeutics
Prince of Wales Hospital
Chinese University of Hong Kong
9
Structure of a clinical trial
1 - Study Sample
10
Structure of a clinical trial
1 - Study Sample
population
Definition of condition
Population with
the Condition
Inclusion Criteria
Population with
the Condition and
Eligibility to be in
The study
People decided not to
be enrolled In the study
Enrollment
Eligible population
Who are enrolled
in the study
Study Sample
11
Structure of a clinical trial
1 - Study Sample
Practice - 1
Identify population of the study?
What was the definition of condition?
What were the inclusion and exclusion criteria?
How many people did decline to
participate and why?
12
Practice 1
Who were enrolled in the study?
Population : Patients in Prince of Wales Hospital in Hong Kong
Definition of condition
Population with : Patients with Stable Angina on low dose
the Condition
Aspirin with a history of GI bleeding
Inclusion & Exclusion Criteria
Population with
the Condition and : Age ≤ 70, Prior history of GI bleeding,
Use of other NSAIDS, HP positive, …
Eligibility to be in
The study
Enrollment
Study Sample
13
Structure of a clinical trial
2 - Randomization
14
Structure of a clinical trial
2 - Randomization
Study Sample
Intervention Group
Control Group
15
Structure of a clinical trial
2 - Randomization
What is Randomization?
Why do we use Randomization?
What type of Randomization
Should we use?
16
Structure of a clinical trial
2 - Randomization
What is Randomization?
It is a process by which all participants
are equally likely to be assigned to either
the intervention or control group
17
Structure of a clinical trial
2 - Randomization
Why do we use Randomization?
Precision & Accuracy
Error & Bias
http://dl.clackamas.cc.or.us/ch104-01/accuracy_vs_precision.htm
18
Structure of a clinical trial
2 - Randomization
Why do we use Randomization?
•
•
Minimize selection bias
Maximize statistical power
19
Structure of a clinical trial
2 - Randomization
Types of Randomization
Simple/ Complete/ Unrestricted
Randomization
Robust against selection
Its drawback is imbalance group size
In small RCTs ≤ 200
20
Computer generated random number
21
Structure of a clinical trial
2 - Randomization
Types of Randomization
Restricted Randomization
Adaptive Randomization
…
Balance groups size
Increase the chance of selection bias
Difficulty in Analyses
22
Structure of a clinical trial
2 - Randomization
Practice - 2
Did they randomize the patients?
What was the Randomization Method?
23
Structure of a clinical trial
3 – Allocation Concealment
24
Structure of a clinical trial
3 – Allocation Concealment
What is allocation concealment?
Concealment of allocation sequence
From those assigning participants
25
26
Structure of a clinical trial
3 – Allocation Concealment
Types of allocation concealment
Sequentially controlled Numbered, Opaque
Sealed Envelopes (SNORE)
Pharmacy controlled numbered/ coded
Containers
Central Randomization
…..
27
Structure of a clinical trial
3 – Allocation Concealment
Practice - 3
Did they conceal the allocation ?
What was the allocation concealment Method?
Were the study groups comparable before
intervention?
28
Structure of a clinical trial
4 – Blinding
29
Structure of a clinical trial
4 – Blinding
What is blinding?
Methods that prevent study participants,
caregivers, or those assessing outcomes
from knowing which intervention was
received in each group
30
Structure of a clinical trial
4 – Blinding
Issues with Blinding
Sometimes blinding is impossible.
Investigator should say who was blinded
Unblinded RCTs (Objective/ Subjective outcomes)
31
Structure of a clinical trial
4 – Blinding
Practice - 4
Did they use any blinding method?
What was the blinding Method?
Were study groups comparable
during intervention?
32
Structure of a clinical trial
Randomization
Allocation Concealment
Follow up
Intervention
Group
Study Sample
Measuring Outcomes
Blinding
Control Group
Loss to Follow up
Poor compliance
Contamination
33
Structure of a clinical trial
5 - Issues during follow up
“Loss to follow up”
What would be the reasons behind loss to follow up?
• Treatment side effect
• Moved away, died
• Got worse or better
• Lost of interest
34
Structure of a clinical trial
5 - Issues during follow up
“Loss to follow up”
What would be the impact of loss to follow up on
results?
If it affects one group more than the other, the results will be biased
If it happens in high rate,
Their outcomes can’t be included in analyses
It is like having smaller sample size
Random Error will increase
Accuracy and study power will decrease
35
Structure of a clinical trial
5 - Issues during follow up
“Loss to follow up”
How to deal with loss to follow up?
Impute an outcome based on missing data protocol.
Using last or worst observation
Predict the un-observed outcome
‘5 and 20 rule” of study population
Best case, worst case sensitivity analysis
Screening visits before enrollment (weeds out the time wasters)
Pre-randomization run in method
No matter what techniques have been
used the results have to be used cautiously.
36
Structure of a clinical trial
5 - Issues during follow up
“poor compliance”
Poor compliance may occur for the same reasons
People with poor compliance tend to have worse prognosis
Clofibrate Trial
Mortality (Cum – 5 y)
Treatment group
Control group
Compliant
15%
15.1%
Non-compliant
24.6%
28.3%
Mathew D. Reeves, PhD
@Department of Epidemiology, MSU
37
Structure of a clinical trial
5 - Issues during follow up
“Contamination”
What is contamination?
Situation when subjects cross-over from
one arm into the other one
Why contamination is important?
It is like not having randomization.
It creates selection bias
38
Structure of a clinical trial
5 - Issues during follow up
Practice - 5
Was there any loss to follow up?
What was any poor compliance?
Was there any contamination?
39
Intervention
Group
200
Poor Compliance
40
165
Study Sample
400
Control
Group
200
Measuring Outcomes
Structure of a clinical trial
6 – Analysis
165
Three main approaches:
Intention to Treat Analysis (ITT)
Per Protocol Analysis (PPA)
As Treated Analysis (ATA)
40
Intervention Poor Compliance
Group
40
200
165
Study Sample
400
Aspirin + PPI
Group
200
Measuring Outcomes
Structure of a clinical trial
6 – Analysis, Intention To Treat (ITT)
165
ITT compares outcome based on the original treatment arm that each individual
participants was randomized to regardless of protocol violation.
ITT gives the most valid but conservative estimate of the true treatment effect.
When there is protocol violation ITT will not estimate true treatment effect.
41
Intervention
Group
200
Poor Compliance
40
160
Study Sample
400
Aspirin + PPI
Group
200
Measuring Outcomes
Structure of a clinical trial
6 – Analysis, Per Protocol Analysis (PPA)
165
- In PPA Just people who complied with the original randomization are analyzed.
- The treatment effect can be just apply to those people who complied to protocol.
- We can never get an unbiased assessment of the true treatment effect.
42
Intervention
Group
200
Poor Compliance
40
165
Study Sample
400
Aspirin + PPI
Group
200
Measuring Outcomes
Structure of a clinical trial
6 – Analysis, As Treated Analysis (ATA)
235
Participants are analyzed based on whether they got the treatment or not
regardless of the original group (It completely kills the randomization).
43
Structure of a clinical trial
6 - Analysis
Practice - 6
What kind of analysis approach did they use?
Was it a good idea to use that approach?
44
Intervention
200
Study Sample
400
Control
200
20
(10%)
Measuring Outcomes
GI Bleeding
Structure of a clinical trial
7 – Analysis (measuring outcome)
How large is the treatment effect?
40
(20%)
How might we express the results of dichotomous outcome?
- Absolute risk reduction (ARR)
- Relative risk reduction (RRR)
- Number Needed to Treat (NNT)
45
Structure of a clinical trial
7 – Analysis (measuring outcome)
How large is the treatment effect?
Measuring Outcomes
GI Bleeding
Intervention
200
20
(10%)
Study Sample
400
Control
200
40
(20%)
Outcome (GI Bleeding)
Exposure
Total
Intervention
20
180
200
Control
40
160
200
46
Structure of a clinical trial
7 – Analysis (measuring outcome)
How large is the treatment effect?
Outcome (GI Bleeding)
Exposure
Total
Intervention
20
180
200
Control
40
160
200
Absolute Risk (Probability) of Bleeding in control group (CGR): 40/200=20%
Absolute Risk (Probability) of Bleeding in intervention group (IGR): 20/200=10%
Absolute Risk Reduction (ARR) = CGR – IGR = 20% - 10% = 10%
(The risk of bleeding in Pt. treated with intervention is 10% lower than the risk in Pt.
On standard treatment)
Relative Risk (RR) = IGR/ CGR = 10/20 = 0.5
Relative Risk Reduction (RRR) = 1 – RR = 1 – 50%= 50%
(Intervention decreases the risk of Bleeding by about 50% compared to standard
treatment)
47
Structure of a clinical trial
7 – Analysis (measuring outcome)
How large is the treatment effect?
Outcome (GI Bleeding)
Exposure
Total
Intervention
20
180
200
Control
40
160
200
Number Needed to Treat (NNT)
Number of Pt. needed to be treated to prevent one adverse effect
NNT = 1 / ARR
NNT = 1 / ARR = 1/ 0.1 = 10 (during specific time)
48
Outcome
Statistical method
Categorical
Chi-Square
Continues
T-Test
Not normally distributed
Non parametric test
Survival
Kaplan-Meire/ Cox Regression
49
Structure of a clinical trial
8 – Analysis
How precise is the estimate of the treatment effect?
We want to know how close is our point estimate to true treatment effect
Treatment effect (mn)
Study sample (n)
Study sample (1)
Treatment effect (m1)
Population with
Condition&Eligibility
True treatment
Effect (M)
Study sample (2)
Treatment effect (m2)
To assess the precision of the estimate we have to know
about Confidence Interval
50
Confidence Interval
Small confidence
interval
Treatment effect (m1)
Big confidence
interval
51