1. No category

Heterogeneity in Meta-Analysis
Subgroup Analysis
Meta-Regression
Discussion
References
Heterogeneity in Meta-Analysis
Subgroup Analysis
Meta-Regression
Discussion
References
Plan of the Session
Session II: Heterogeneity in Meta-Analysis
The aim of this session is to:
James Carpenter1 , Ulrike Krahn2,3 , Gerta R¨
ucker4 , Guido Schwarzer4
1 London
School of Hygiene and Tropical Medicine & MRC Clinical Trials Unit, London, UK
2 Institute of Medical Biostatistics, Epidemiology and Informatics, Mainz, Germany
3 Institute of Medical Informatics, Biometry and Epidemiology, Duisburg-Essen, Germany
4 Institute for Medical Biometry and Statistics, Freiburg, Germany
[email protected]
1
Subgroup Analysis
introduce random effects model and sources of heterogeneity, and its
consequences;
I
discuss how to detect heterogeneity;
I
introduce statistical methods which attempt to allow for for
heterogeneity (e.g. subgroup analysis and meta-regression), and
I
illustrate with examples.
At the end of this session the objectives are that you should
IBC Short Course
Florence, 6 July 2014
Heterogeneity in Meta-Analysis
I
Meta-Regression
Discussion
References
I
understand the sources, and potential consequences of heterogeneity;
I
understand, and be able to perform subgroup analysis and
meta-regression, and
I
be able to summarise the results for a clinical audience.
Carpenter/Krahn/R¨
ucker/Schwarzer
Heterogeneity in Meta-Analysis
Random effects model
Session II: Heterogeneity in Meta-Analysis
Subgroup Analysis
Meta-Regression
Florence, 6 July 2014
Discussion
2
References
Random Effects Model – Graphical Presentation
Average effect
DerSimonian and Laird (1986); Fleiss (1993):
iid
θˆk ∼ N(θk , σ2k )
I
I
I
Study 1
θˆk observed treatment effect in study i
θk true mean in study i, σk standard error in study i
True study means θk vary randomly around a fixed global mean θ with
a between-study (heterogeneity) variance τ2 :
Study 2
Study 3
iid
θk ∼ N(θ, τ2 )
Alternative notation
Study 4
q
θˆk = θ + σ2k + τ2 k ,
I
iid
k ∼ N(0, 1)
Study 5
Two variance components: σ2k within-study sampling variance,
τ2 between-study (heterogeneity) variance
0.1
0.2
0.5
1
2
5
10
Odds ratio
Carpenter/Krahn/R¨
ucker/Schwarzer
Session II: Heterogeneity in Meta-Analysis
Florence, 6 July 2014
3
Carpenter/Krahn/R¨
ucker/Schwarzer
Session II: Heterogeneity in Meta-Analysis
Florence, 6 July 2014
4
Heterogeneity in Meta-Analysis
Subgroup Analysis
Meta-Regression
Discussion
References
Heterogeneity in Meta-Analysis
Random Effects Model – Graphical Presentation
Subgroup Analysis
Meta-Regression
Discussion
References
Inverse variance weighting (both models)
I
Pooled effect estimate: Weighted mean of the study estimates
P ˆ
wk θk
θˆ = P
wk
I
Weights in fixed effect model:
Study 1
Study 2
wk =
Study 3
I
1
σ
ˆ 2k
Weights in random effects model:
Study 4
wk =
Study 5
I
0.1
0.2
0.5
1
2
5
10
σ
ˆ 2k
1
+ ˆτ2
Large heterogeneity ⇒ Weights tend to be more similar ⇒ Smaller
studies get larger weights
Odds ratio
Carpenter/Krahn/R¨
ucker/Schwarzer
Heterogeneity in Meta-Analysis
Session II: Heterogeneity in Meta-Analysis
Subgroup Analysis
Meta-Regression
Florence, 6 July 2014
Discussion
5
References
Carpenter/Krahn/R¨
ucker/Schwarzer
Heterogeneity in Meta-Analysis
Overall Survival – Forestplot (example from first talk)
Session II: Heterogeneity in Meta-Analysis
Subgroup Analysis
Meta-Regression
Florence, 6 July 2014
Discussion
6
References
Comparison of Random and Fixed Effect Model
> forest(m1, hetstat=FALSE, comb.random=TRUE)
Study
De Souza
Gianni
Gisselbrecht
Haioun
Intragumtornchai
Kaiser
Kluin−Nelemans
Martelli
Martelli 2003
Milpied
Rodriguez 2003
Santini
Verdonck
Vitolo
TE
seTE
−0.08
−0.65
0.37
−0.04
−0.45
0.08
0.20
−0.38
0.01
−0.44
0.29
−0.21
0.34
0.34
0.3672
0.3850
0.1487
0.1529
0.3852
0.1834
0.2697
0.4473
0.2748
0.2481
0.3482
0.2697
0.3290
0.2749
Hazard Ratio
HR
0.92
0.52
1.45
0.96
0.64
1.08
1.23
0.69
1.01
0.64
1.34
0.81
1.40
1.41
Fixed effect model
Random effects model
0.5
Carpenter/Krahn/R¨
ucker/Schwarzer
1
95%−CI W(fixed) W(random)
[0.45; 1.89]
[0.24; 1.11]
[1.08; 1.93]
[0.71; 1.30]
[0.30; 1.36]
[0.75; 1.55]
[0.72; 2.08]
[0.29; 1.65]
[0.59; 1.73]
[0.40; 1.05]
[0.68; 2.65]
[0.48; 1.37]
[0.73; 2.67]
[0.82; 2.41]
3.2%
2.9%
19.5%
18.5%
2.9%
12.9%
5.9%
2.2%
5.7%
7.0%
3.6%
5.9%
4.0%
5.7%
4.3%
4.0%
14.2%
13.8%
4.0%
11.5%
7.0%
3.1%
6.8%
7.9%
4.7%
7.0%
5.1%
6.8%
1.05 [0.92; 1.19]
1.02 [0.86; 1.20]
100%
−−
−−
100%
I
Random effects model: Effects vary randomly around a global mean
I
Exchangeability: Effects are assumed to be exchangeable a-priori
(Higgins et al., 2009)
I
I
I
I
Study-treatment interactions (contrasts) interpreted as a random
sample from a ‘population of potential treatment effects’ (Senn, 2000,
2010)
Confidence intervals for random effects model at least as wide as those
for fixed effect model
In addition, Higgins et al. (2009) proposed a prediction interval that
gives a range of effects of future studies
Fixed effect model
I
Special case of random effects model for τ2 = 0
⇒ Often no difference in pooled estimates since often τ2 ≈ 0
2
Session II: Heterogeneity in Meta-Analysis
Florence, 6 July 2014
7
Carpenter/Krahn/R¨
ucker/Schwarzer
Session II: Heterogeneity in Meta-Analysis
Florence, 6 July 2014
8
Heterogeneity in Meta-Analysis
Subgroup Analysis
Meta-Regression
Discussion
References
Heterogeneity in Meta-Analysis
Sources of Heterogeneity
I
I
Clinical baseline heterogeneity between patients from different
studies (e.g. baseline characteristics); not necessarily reflected on
outcome measurement scale
I
Heterogeneity from other sources, e.g. design-related heterogeneity
I
Heterogeneity reporting and related bias, e.g. selective publication
of small studies if they show an effect
I
I
Subgroup Analysis
I
I
Discussion
9
References
High risk vs low risk patients
High quality vs low quality studies
Meta-regression
I
Florence, 6 July 2014
Meta-Regression
Use categorical variable (with limited number of possible values) to
define subgroups – ideally pre-specified variable
Explains heterogeneity well if large variation exists between subgroup
means and little variation within subgroups
Examples:
I
I
Session II: Heterogeneity in Meta-Analysis
Adjusts for unexplained heterogeneity (via wider confidence interval)
Does not explain heterogeneity
I
Statistical heterogeneity quantified on the outcome measurement
scale (may or may not be clinically relevant and may or may not be
statistically significant)
⇒ Random effects modelling, subgroup analysis, meta-regression
Heterogeneity in Meta-Analysis
Extension of subgroup analysis by allowing for continuous variables
Example: Geographical latitude as effect modifier for efficacy of vaccine
in tuberculosis prevention (Colditz et al., 1994)
Risk of ecological bias if continuous variable measured on study-level
(e.g. mean age) (Higgins and Thompson, 2004)
Carpenter/Krahn/R¨
ucker/Schwarzer
Heterogeneity in Meta-Analysis
Session II: Heterogeneity in Meta-Analysis
Subgroup Analysis
Florence, 6 July 2014
Meta-Regression
Discussion
Example: Amiodarone for Conversion of Atrial Fibrillation
Amiodarone example – Forest plot
Letelier et al. (2003), Arch Intern Med:
> forest(m1, leftcols="studlab", xlim=c(0.05, 20),
+
print.tau2=TRUE, print.I2=FALSE, print.pval.Q=TRUE)
I
21 randomised or quasi-randomised controlled trials
Patients with atrial fibrillation (AF) of any etiology and duration
I
Amiodarone compared with placebo, digoxin, CCB, or no treatment
I
Primary outcome: conversion to sinus rhythm within 4 weeks (binary
outcome)
Cowan 1986
Noc 1990
Capucci 1992
Cochrane 1994
Hou 1995
Kondili 1995
Donovan 1994
Galve 1996
Kontoyannis 1998
Bellandi 1999
Kochidiakis 1999
Cotter 1999
Peuhkurinen 2000
Vardas 2000
Joseph 2000
Cybulski 2001
Galperin 2000
Bianconi 2000
Villani 2000
Hohnloser 2000
Natale 2000
> amio <- read.table(file="Amiodarone.csv", header=TRUE, sep = ",",
+
quote="\"", dec=".", fill = TRUE)
> names(amio)
>
>
>
>
"afduration" "RR"
"LL"
"UL"
"N"
amio$logRR <- log(amio$RR)
# Recalculation of standard errors
amio$SE <- (log(amio$UL)-log(amio$LL))/2/1.96
m1 <- metagen(logRR, SE, data=amio, sm="RR", studlab=Study)
Carpenter/Krahn/R¨
ucker/Schwarzer
Session II: Heterogeneity in Meta-Analysis
Risk Ratio
Study
I
[1] "Study"
References
Subgroup analysis
I
I
Carpenter/Krahn/R¨
ucker/Schwarzer
Discussion
Random effects model
Principal sources of heterogeneity in meta-analysis:
I
Meta-Regression
Explaining heterogeneity in meta-analysis
I
I
Subgroup Analysis
Florence, 6 July 2014
RR
1.11
18.00
0.77
1.15
1.29
1.33
1.05
1.13
1.42
1.41
1.46
1.43
2.45
2.01
1.32
1.87
33.70
2.04
4.75
3.13
5.12
Fixed effect model
Random effects model
10
References
95%−CI W(fixed) W(random)
[0.78; 1.58]
[1.17; 276.45]
[0.37; 1.61]
[0.91; 1.45]
[0.97; 1.72]
[0.71; 2.48]
[0.69; 1.60]
[0.84; 1.52]
[1.08; 1.86]
[1.15; 1.72]
[1.19; 1.79]
[1.14; 1.79]
[1.49; 4.02]
[1.55; 2.60]
[0.96; 1.82]
[1.37; 2.55]
[2.08; 545.97]
[0.19; 21.95]
[1.61; 14.05]
[1.48; 6.62]
[2.61; 10.04]
4.3%
0.1%
1.0%
10.1%
6.5%
1.4%
3.0%
6.0%
7.3%
13.1%
13.1%
10.6%
2.2%
7.9%
5.2%
5.5%
0.1%
0.1%
0.5%
0.9%
1.2%
5.9%
0.3%
2.7%
7.4%
6.7%
3.4%
5.1%
6.6%
6.9%
7.8%
7.8%
7.5%
4.4%
7.1%
6.3%
6.4%
0.3%
0.3%
1.5%
2.6%
3.0%
1.45 [1.35; 1.56]
1.53 [1.32; 1.76]
100%
−−
−−
100%
Heterogeneity: tau−squared=0.0581, p<0.0001
0.05
11
0.5 1
Carpenter/Krahn/R¨
ucker/Schwarzer
2
10 20
Session II: Heterogeneity in Meta-Analysis
Florence, 6 July 2014
12
Heterogeneity in Meta-Analysis
Subgroup Analysis
Meta-Regression
Discussion
References
Amiodarone example – Inverse Variance Method
Heterogeneity in Meta-Analysis
Subgroup Analysis
Meta-Regression
Discussion
References
Amiodarone example – Subgroup Analysis
> print(summary(m1), digits=2)
A priori specified subgroup analyses:
Number of studies combined: k=21
1. Mean duration of the current AF episode (≤ 48 vs > 48 hours)
RR
95%-CI
z p.value
Fixed effect model
1.45 [1.35; 1.56] 9.93 < 0.0001
Random effects model 1.53 [1.32; 1.76] 5.77 < 0.0001
2. Mean left atrial size (< 45 vs ≥ 45 mm)
3. Proportion of patients with underlying cardiovascular disease (< 50%
vs ≥ 50%)
Quantifying heterogeneity:
tau^2 = 0.0581; H = 1.71 [1.36; 2.15]; I^2 = 65.8% [45.8%; 78.4%]
4. Type of control treatment (placebo or no treatment vs digoxin vs
CCB)
Test of heterogeneity:
Q d.f. p.value
58.46
20 < 0.0001
5. Amiodarone regimen (single vs continuous oral or intravenous dosage)
6. Time to outcome measurement (< 12 vs ≥ 12 hours)
> m2 <- update(m1, byvar=afduration, print.byvar=FALSE)
> m3 <- update(m1, byvar=afduration, print.byvar=FALSE, tau.common=TRUE)
Details on meta-analytical method:
- Inverse variance method
- DerSimonian-Laird estimator for tau^2
Carpenter/Krahn/R¨
ucker/Schwarzer
Heterogeneity in Meta-Analysis
Session II: Heterogeneity in Meta-Analysis
Subgroup Analysis
Meta-Regression
Florence, 6 July 2014
Discussion
13
References
Notation for Subgroup Analysis (like Analysis of Variance)
I
I
I
wsk θˆsk
k=1
I
I
I
wsk
Weight wsk is the inverse variance of study k in subgroup s
P s
Denominator is the subgroup weight Ws = K
w
k=1 sk
Global mean
Pp
Meta-Regression
PKs
ˆ
s=1 k=1 wsk θsk
θˆ = P
PKs
p
s=1 k=1 wsk
Session II: Heterogeneity in Meta-Analysis
Florence, 6 July 2014
15
RR
Discussion
14
References
95%−CI
AF Duration <= 48h
Cowan 1986
Noc 1990
Capucci 1992
Cochrane 1994
Hou 1995
Kondili 1995
Donovan 1994
Galve 1996
Kontoyannis 1998
Bellandi 1999
Kochidiakis 1999
Cotter 1999
Peuhkurinen 2000
Vardas 2000
Joseph 2000
Cybulski 2001
Fixed effect model
Random effects model
1.11 [0.78; 1.58]
18.00 [1.17; 276.45]
0.77 [0.37; 1.61]
1.15 [0.91; 1.45]
1.29 [0.97; 1.72]
1.33 [0.71; 2.48]
1.05 [0.69; 1.60]
1.13 [0.84; 1.52]
1.42 [1.08; 1.86]
1.41 [1.15; 1.72]
1.46 [1.19; 1.79]
1.43 [1.14; 1.79]
2.45 [1.49; 4.02]
2.01 [1.55; 2.60]
1.32 [0.96; 1.82]
1.87 [1.37; 2.55]
1.40 [1.30; 1.51]
1.40 [1.25; 1.57]
AF Duration > 48h
Galperin 2000
Bianconi 2000
Villani 2000
Hohnloser 2000
Natale 2000
Fixed effect model
Random effects model
33.70
2.04
4.75
3.13
5.12
4.33
4.33
0.05
Carpenter/Krahn/R¨
ucker/Schwarzer
Subgroup Analysis
Risk Ratio
Study
Weighted mean estimate for subgroup s (s = 1, . . . , p):
k=1
θˆs = P
Ks
Heterogeneity in Meta-Analysis
Florence, 6 July 2014
> forest(m2, leftcols="studlab", xlim=c(0.05, 20),
+
hetstat=FALSE, overall=FALSE, col.by="darkblue")
Within subgroup s, Ks studies (k = 1, . . . , Ks )
P
K = ps=1 Ks total number of studies over all subgroups
PKs
Session II: Heterogeneity in Meta-Analysis
Amiodarone example – Forest plot with subgroups
p subgroups of studies (s = 1, . . . , p)
I
Carpenter/Krahn/R¨
ucker/Schwarzer
0.5 1
Carpenter/Krahn/R¨
ucker/Schwarzer
2
[2.08; 545.97]
[0.19; 21.95]
[1.61; 14.05]
[1.48; 6.62]
[2.61; 10.04]
[2.79; 6.73]
[2.79; 6.73]
10 20
Session II: Heterogeneity in Meta-Analysis
Florence, 6 July 2014
16
Heterogeneity in Meta-Analysis
Subgroup Analysis
Meta-Regression
Discussion
References
Heterogeneity Statistics
I
Heterogeneity in Meta-Analysis
Subgroup Analysis
Meta-Regression
Discussion
References
Partition of Q (in Fixed Effect Model)
Between-groups heterogeneity statistic QB :
QB =
p
X
I
QW can be partitioned into subgroup-specific parts QWs by
2
Ws θˆs − θˆ
QW =
s=1
p
X
QWs ,
s=1
I
Under homogeneity between subgroups, QB follows a χ2 -distribution
with p − 1 degrees of freedom
each with Ks − 1 degrees of freedom
I
I
Within-group heterogeneity statistic QW :
QW =
p X
Ks
X
I
I
2
wsk θˆsk − θˆs
with (p − 1) + (K − p) = K − 1 degrees of freedom
(Cooper et al., 2009)
Under homogeneity within all subgroups, QW follows a χ2 -distribution
with K − p degrees of freedom
Carpenter/Krahn/R¨
ucker/Schwarzer
Heterogeneity in Meta-Analysis
Session II: Heterogeneity in Meta-Analysis
Subgroup Analysis
In total, we have
Q = QB + QW
s=1 k=1
I
P
P
QW has ps=1 (Ks − 1) = ps=1 Ks − p = K − p degrees of freedom
QWs measures the extent of heterogeneity attributed to subgroup s
Florence, 6 July 2014
Meta-Regression
Discussion
17
References
Carpenter/Krahn/R¨
ucker/Schwarzer
Heterogeneity in Meta-Analysis
Session II: Heterogeneity in Meta-Analysis
Subgroup Analysis
Florence, 6 July 2014
Meta-Regression
Amiodarone example – Partition of Q
Amiodarone example: AF subgroup results
> print(summary(m2, comb.random=FALSE), digits=2)
Fixed effect model (Qb = 24.42, df = 1, p < 0.0001,
Qw = 34.04, df = 19, p = 0.018):
*** Output truncated ***
Subgroup
AF Duration ≤ 48h
AF Duration > 48h
Test of heterogeneity:
Q d.f. p.value
58.46
20 < 0.0001
RR
1.40
4.33
95%-CI
[1.30; 1.51]
[2.79; 6.73]
Q
30.58
3.46
References
ˆτ2
0.0243
0
I2
50.9%
0%
ˆτ2
0.0243
0
I2
50.9%
0%
Random effects model (Qb = 23.69, df = 1, p < 0.0001):
Results for subgroups (fixed effect model):
k
RR
95%-CI
Q
AF Duration <= 48h 16 1.40 [1.30; 1.51] 30.58
AF Duration > 48h
5 4.33 [2.79; 6.73] 3.46
Subgroup
AF Duration ≤ 48h
AF Duration > 48h
tau^2
I^2
0.0243 50.9%
0
0%
Session II: Heterogeneity in Meta-Analysis
k
16
5
RR
1.40
4.33
95%-CI
[1.25; 1.57]
[2.79; 6.73]
Q
30.58
3.46
Random effects model with common τ2 (Qb = 20.92, df = 1, p < 0.0001):
Test for subgroup differences (fixed effect model):
Q d.f. p.value
Between groups 24.42
1 < 0.0001
Within groups 34.04
19
0.0182
Carpenter/Krahn/R¨
ucker/Schwarzer
k
16
5
Discussion
18
Subgroup
AF Duration ≤ 48h
AF Duration > 48h
Florence, 6 July 2014
19
Carpenter/Krahn/R¨
ucker/Schwarzer
k
16
5
RR
1.40
4.34
95%-CI
[1.25; 1.57]
[2.71; 6.96]
Session II: Heterogeneity in Meta-Analysis
Q
30.58
3.46
ˆτ2
0.023
0.023
I2
50.9%
0%
Florence, 6 July 2014
20
Heterogeneity in Meta-Analysis
Subgroup Analysis
Meta-Regression
Discussion
References
Meta-Regression
Meta-Regression
Discussion
References
> mr2 <- metareg(m2)
> print(mr2)
Mixed-Effects Model (k = 21; tau^2 estimator: DL)
i.i.d.
θˆk = θ + βxk + σ
ˆ k k ,
k ∼ N(0, 1),
k = 1, . . . , K
*** Output truncated ***
Test for Residual Heterogeneity:
QE(df = 19) = 34.0399, p-val = 0.0182
Random effects meta-regression:
θˆk = θ + βxk + uk + σk k ,
i.i.d.
i.i.d.
k ∼ N(0, 1); uk ∼ N(0, τ2 )
Test of Moderators (coefficient(s) 2):
QM(df = 1) = 20.9197, p-val < .0001
with covariate xk and regression parameter β
Model Results:
Meta-regression of AF duration in amiodarone example:
I
xk = 1 if AF duration > 48 hours
I
xk = 0 if AF duration ≤ 48 hours
Heterogeneity in Meta-Analysis
Subgroup Analysis
Amiodarone example – Meta-regression
Fixed effect meta-regression:
Carpenter/Krahn/R¨
ucker/Schwarzer
Heterogeneity in Meta-Analysis
0.3353
1.1330
se
0.0571
0.2477
zval
5.8746
4.5738
pval
<.0001
<.0001
ci.lb
0.2234
0.6475
ci.ub
0.4471
1.6185
ˆ = exp(0.3353) = 1.40
Risk Ratio (AF ≤ 48 hours): exp(θ)
ˆ + β)
ˆ = exp(0.3353 + 1.1330) = 4.34
I Risk Ratio (AF > 48 hours): exp(θ
I
Session II: Heterogeneity in Meta-Analysis
Subgroup Analysis
intrcpt
AF Duration > 48h
Meta-Regression
Florence, 6 July 2014
Discussion
21
References
Amiodarone example – Bubble plot
Carpenter/Krahn/R¨
ucker/Schwarzer
Heterogeneity in Meta-Analysis
Session II: Heterogeneity in Meta-Analysis
Subgroup Analysis
Meta-Regression
Florence, 6 July 2014
Discussion
22
References
Example: Vaccine for the prevention of tuberculosis
> bubble(mr2, lwd=2, col.line="blue")
●
3
Colditz et al. (1994), JAMA:
Efficacy of vaccine for the prevention of tuberculosis
I
13 prospective trials
2
I
●
●
> data(dat.colditz1994, package="metafor")
> m4 <- metabin(tpos, tpos+tneg, cpos, cpos+cneg,
+
data=dat.colditz1994, studlab=paste(author, year))
> forest(m4, rightcols=c("effect", "ci"), print.pval.Q=FALSE,
+
label.left="Vaccine better", label.right="Vaccine worse")
●
1
Treatment effect (log risk ratio)
●
●
●
0
●
●
●
AF Duration <= 48h
AF Duration > 48h
Covariate afduration
Carpenter/Krahn/R¨
ucker/Schwarzer
Session II: Heterogeneity in Meta-Analysis
Florence, 6 July 2014
23
Carpenter/Krahn/R¨
ucker/Schwarzer
Session II: Heterogeneity in Meta-Analysis
Florence, 6 July 2014
24
Heterogeneity in Meta-Analysis
Subgroup Analysis
Meta-Regression
Discussion
References
Vaccination example – Forest plot
Study
Aronson 1948
Ferguson & Simes 1949
Rosenthal et al 1960
Hart & Sutherland 1977
Frimodt−Moller et al 1973
Stein & Aronson 1953
Vandiviere et al 1973
TPT Madras 1980
Coetzee & Berjak 1968
Rosenthal et al 1961
Comstock et al 1974
Comstock & Webster 1969
Comstock et al 1976
4
6
3
62
33
180
8
505
29
17
186
5
27
Fixed effect model
Random effects model
123
306
231
13598
5069
1541
2545
88391
7499
1716
50634
2498
16913
11
29
11
248
47
372
10
499
45
65
141
3
29
191064
139
303
220
12867
5808
1451
629
88391
7277
1665
27338
2341
17854
166283
RR
95%−CI
0.41
0.20
0.26
0.24
0.80
0.46
0.20
1.01
0.63
0.25
0.71
1.56
0.98
[0.13; 1.26]
[0.09; 0.49]
[0.07; 0.92]
[0.18; 0.31]
[0.52; 1.25]
[0.39; 0.54]
[0.08; 0.50]
[0.89; 1.14]
[0.39; 1.00]
[0.15; 0.43]
[0.57; 0.89]
[0.37; 6.53]
[0.58; 1.66]
I
I
I
I
I
I
I
0.64 [0.59; 0.69]
0.49 [0.34; 0.70]
Session II: Heterogeneity in Meta-Analysis
Subgroup Analysis
Meta-Regression
Florence, 6 July 2014
Discussion
25
References
Vaccination example – Forestplot (Sorted by Latitude)
Study
Frimodt−Moller et al 1973
TPT Madras 1980
Comstock et al 1974
Vandiviere et al 1973
Coetzee & Berjak 1968
Comstock & Webster 1969
Comstock et al 1976
Rosenthal et al 1960
Rosenthal et al 1961
Aronson 1948
Stein & Aronson 1953
Hart & Sutherland 1977
Ferguson & Simes 1949
Latitude
Risk Ratio
13
13
18
19
27
33
33
42
42
44
44
52
55
Fixed effect model
Random effects model
References
Absolute geographical latitude (i.e. distance from the equator)
Study validity score
Mean age at vaccination
Study design
Year of publication
Year trial began
Follow-up time (in years)
RR
95%−CI
0.80
1.01
0.71
0.20
0.63
1.56
0.98
0.26
0.25
0.41
0.46
0.24
0.20
[0.52; 1.25]
[0.89; 1.14]
[0.57; 0.89]
[0.08; 0.50]
[0.39; 1.00]
[0.37; 6.53]
[0.58; 1.66]
[0.07; 0.92]
[0.15; 0.43]
[0.13; 1.26]
[0.39; 0.54]
[0.18; 0.31]
[0.09; 0.49]
Carpenter/Krahn/R¨
ucker/Schwarzer
Heterogeneity in Meta-Analysis
Session II: Heterogeneity in Meta-Analysis
Subgroup Analysis
Florence, 6 July 2014
Meta-Regression
Discussion
26
References
Vaccination example – Meta-Regression
Random effects meta-regression:
θˆk = θ + βxk + uk + σk k ,
i.i.d.
i.i.d.
k ∼ N(0, 1); uk ∼ N(0, τ2 )
where covariate xk is the absolute geographical latitude
Results:1
Intercept
Latitude
Estimate
SE
z-value
p-value
θˆ = 0.2595
βˆ = −0.0292
0.2323
0.0067
1.12
-4.34
0.264
< 0.0001
0.64 [0.59; 0.69]
0.49 [0.34; 0.70]
ˆ
τ2 = 0.0633 (estimated according to DerSimonian and Laird (1986))
Parameter β is a log risk ratio
Heterogeneity: I−squared=92.1%, tau−squared=0.3095
0.1
0.5 1 2
10
Vaccine better Vaccine worse
Carpenter/Krahn/R¨
ucker/Schwarzer
Discussion
> forest(m4, sortvar=m4$data$ablat,
+
leftcols=c("studlab", "ablat"), leftlabs=c(NA, "Latitude"),
+
rightcols=c("effect", "ci"), print.pval.Q=FALSE,
+
label.left="Vaccine better", label.right="Vaccine worse")
0.1
0.5 1 2
10
Vaccine better Vaccine worse
Heterogeneity in Meta-Analysis
Meta-Regression
Colditz et al. (1994), JAMA:
I Efficacy of vaccine for the prevention of tuberculosis
I 13 prospective trials
I Random effects meta-regression
I Variables considered in meta-regression:
Heterogeneity: I−squared=92.1%, tau−squared=0.3095
Carpenter/Krahn/R¨
ucker/Schwarzer
Subgroup Analysis
Example: Vaccine for the prevention of tuberculosis
Risk Ratio
Experimental
Control
Events Total Events
Total
Heterogeneity in Meta-Analysis
Session II: Heterogeneity in Meta-Analysis
1
Florence, 6 July 2014
27
R command: mr4 <- metareg(m4, ablat)
Carpenter/Krahn/R¨
ucker/Schwarzer
Session II: Heterogeneity in Meta-Analysis
Florence, 6 July 2014
28
Heterogeneity in Meta-Analysis
Subgroup Analysis
Meta-Regression
Discussion
References
Vaccination example – Bubble plot
Heterogeneity in Meta-Analysis
Subgroup Analysis
Meta-Regression
Discussion
References
Caveats with Meta-Regression and Subgroup Analysis
0.5
> bubble(mr4, lwd=2, col.line="blue")
●
−0.5
I
Visual presentation of relationship essential
I
Often meta-regression does not explain all observed heterogeneity
⇒ Meta-regression should be based on random effects model
Risk of spurious findings from meta-regression/subgroup analysis
I
I
I
●
−1.0
Treatment effect (log risk ratio)
0.0
Thompson and Higgins (2002); Higgins and Thompson (2004):
I
I
−1.5
●
●
●
20
30
40
Avoid meta-regression if there are less than five studies
Limit the number of covariates used in meta-regression
Use only pre-specified characteristics; avoid ‘data-dredging’
Be careful to interpret the coefficient estimates correctly. E.g. if the
outcome is the (log) odds ratio, they are (log) odds ratios of odds
ratios.
50
Covariate ablat
Carpenter/Krahn/R¨
ucker/Schwarzer
Heterogeneity in Meta-Analysis
Session II: Heterogeneity in Meta-Analysis
Subgroup Analysis
Meta-Regression
Florence, 6 July 2014
Discussion
29
References
Caveats with Meta-Regression and Subgroup Analysis
I
I
Not for averages of patient characteristics within trials, for example
average age or proportion of women
Not for ‘baseline risk’ = control group risk: Potential regression to the
mean
Meta-regression at individual-patient level possible if individual
patient data are available (Lambert et al., 2002; Riley et al., 2007,
2008; Jones et al., 2009; Riley et al., 2010)
Carpenter/Krahn/R¨
ucker/Schwarzer
Session II: Heterogeneity in Meta-Analysis
Florence, 6 July 2014
Subgroup Analysis
Meta-Regression
Florence, 6 July 2014
Discussion
30
References
I
Understanding and – as far as possible – explaining heterogeneity is
important if we are to have confidence in the results of our
meta-analysis.
I
Decomposition of the heterogeneity statistic, Q, helps pinpoint the
causes.
I
Subgroup analyses and meta-regression can help explain
heterogeneity, but really need to be pre-specified;
I
If heterogeneity is present then the random effects model should be
used instead of the fixed effects model – principally because of the
larger standard error, which reflects the heterogeneity.
I
However, if there is a contextually important difference between the
fixed and random effect estimates, this may indicate that smaller
studies are systematically different, and further investigation is needed
– see the next session.
Use meta-regression only for study-level covariates
I
I
Heterogeneity in Meta-Analysis
Session II: Heterogeneity in Meta-Analysis
Summary
For aggregate data: Possible bias by confounding (Berlin et al., 2002;
Thompson and Higgins, 2002; Higgins and Thompson, 2004)
I
Carpenter/Krahn/R¨
ucker/Schwarzer
31
Carpenter/Krahn/R¨
ucker/Schwarzer
Session II: Heterogeneity in Meta-Analysis
Florence, 6 July 2014
32
Heterogeneity in Meta-Analysis
Subgroup Analysis
Meta-Regression
Discussion
References
References
Session II: Heterogeneity in Meta-Analysis
Florence, 6 July 2014
Subgroup Analysis
Meta-Regression
Discussion
References
Lambert, P. C., Sutton, A. J., Abrams, K. R., and Jones, D. R. (2002). A comparison of
summary patient-level covariates in meta-regression with individual patient data
meta-analysis. Journal of Clinical Epidemiology, 55:86–94.
Berlin, J. A., Santanna, J., Schmid, C. H., Szczech, L. A., and Feldman, H. I. (2002).
Individual patient- versus group-level data meta-regressions for the investigation of
treatment effect modifiers: Ecological bias rears its ugly head. Statistics in Medicine,
21:371–387.
Colditz, G. A., Brewer, T., Berkey, C., Wilson, M., Burdick, E., Fineberg, H., and
Mosteller, F. (1994). Efficacy of BCG vaccine in the prevention of tuberculosis.
Meta-analysis of the published literature. JAMA, 271(9):698–702.
Cooper, H., Hedges, L. V., and Valentine, J. C. (2009). The Handbook of Research
Synthesis and Meta-analysis. Russell Sage Foundation, New York, 2nd edition.
DerSimonian, R. and Laird, N. (1986). Meta-analysis in clinical trials. Controlled
Clinical Trials, 7:177–188.
Fleiss, J. L. (1993). The statistical basis of meta-analysis. Statistical Methods in
Medical Research, 2:121–145.
Higgins, J. P., Thompson, S. G., and Spiegelhalter, D. J. (2009). A re-evaluation of
random-effects meta-analysis. Journal of the Royal Statistical Society, 172:137–159.
Higgins, J. P. T. and Thompson, S. G. (2004). Controlling the risk of spurious findings
from meta-regression. Statistics in Medicine, 23:1663–1682.
Jones, A. P., Riley, R. D., Williamson, P. R., and Whitehead, A. (2009). Meta-analysis
of individual patient data versus aggregate data from longitudinal clinical trials.
Clinical Trials, 6(1):16–27.
Carpenter/Krahn/R¨
ucker/Schwarzer
Heterogeneity in Meta-Analysis
Letelier, L., Udol, K., Ena, J., Weaver, B., and Guyatt, G. (2003). Effectiveness of
amiodarone for conversion of atrial fibrillation to sinus rhythm - a meta-analysis.
Archives of Internal Medicine, 163:777–85.
Riley, R., Lambert, P., Staessen, J., Wang, J., Gueyffier, F., Thijs, L., and Boutitie, F.
(2008). Meta-analysis of continuous outcomes combining individual patient data and
aggregate data. Statistics in Medicine, 27(11):1870–1893.
Riley, R. D., Lambert, P., and Abo-Zaid, G. (2010). Meta-analysis of individual
participant data: rationale, conduct, and reporting. BMJ, 340.
Riley, R. D., Simmonds, M. C., and Look, M. P. (2007). Evidence synthesis combining
individual patient data and aggregate data: a systematic review identified current
practice and possible methods. Journal of Clinical Epidemiology, 60(5):431–439.
Senn, S. (2000). The many modes of meta. Drug Information Journal, 34:535–549.
Senn, S. (2010). Hans van Houwelingen and the art of summing up. Biometrical
Journal, 52(1):85–94.
Thompson, S. G. and Higgins, J. P. T. (2002). How should meta-regression analyses be
undertaken and interpreted? Statistics in Medicine, 21:1559–1573.
32
Carpenter/Krahn/R¨
ucker/Schwarzer
Session II: Heterogeneity in Meta-Analysis
Florence, 6 July 2014
32