How to plan and prepare a Meta-analysis Claudia Rottschy Felix Hoffstaedter

Clinic for Psychiatry, Psychotherapy
and Psychosomatics
How to plan and prepare a
Meta-analysis
Claudia Rottschy
Felix Hoffstaedter
Structural and functional organization
of the brain (INM-1)
Why Meta-Analyses?
Small samples
Compared to other fields of cognitive and social science and
particularly to clinical research
Publication of isolated findings
Due to logistic expenses, additional experiments for
confirmation and extension of findings are rarely performed
Indirect measures of neuronal activity
Reliability is influenced and probably limited by biological,
technical and methodological confounds
Generalisation of context-specific findings
Inference on brain function and pathomechanisms based on
specific differences in this experimental setting
Why Meta-Analyses?
BUT There are many studies
Recent estimate
15.000 fMRI and PET Paper
All report standardised results!
Derrfuss & Mar 2009 Neuroimaging of working memory
Working memory may be investigated by many paradigms
Sternberg, n-back, Delayed match to sample…
These paradigms may be performed on various material
Verbal items, spatial locations, object properties…
Plus, investigators may add aspects like
Distractors, cross-model transfer, emotional aspects…
4 Performing meta-analyses
-­‐ PubMed search
Working-memory:
[“working memory” OR “short term memory”],
“fMRI”, “healthy subjects”
- PubMed search on related topics
- Review Articles
- Reference tracing Performing meta-analyses
General Inclusion-/Exclusion criteria
Voxel-wise whole brain Analysis
No ROI analyses
No small volume correction
No partial coverage (if evident from methods)
Coordinates in MNI or Talairach space
Careful, legends not always indicative
Good to know standard space of software packages
SPM/FSL: MNI / AFNI/BV: Talairach
Have coordinates been transformed?
Performing meta-analyses
Specific Inclusion-/Exclusion criteria
Healthy subjects
No patients (within group-effect for controls OK)
No selected groups (genetics, behavior)
Within subject, between-condition effects
Normal mapping
No intervention (pharmacology, TMS, training etc)
Only fMRI
Performing meta-analyses
Potential Inclusion-/Exclusion criteria
Paradigm type
This determines what will be represented by the meta-analysis
Contrast type
Low-level baseline (rest, fixation cross)
High-level control task (input, output)
Post-hoc classification (remembered vs. forgotten)
Design-correlations (e.g., with memory load)
Covariate-correlations (e.g., with RT)
Interactions (e.g., valence x load)
Also determines what the results will represent
Meta-analysis on working memory
113 Paper (distinct publications)
1653 subjects (count distinct persons)
189 experiments (contrasts = sets of coordinates)
2662 activations (total number of foci)
Organize your data !
Paper
Subjects
Allen et al 2006
Altamura et al 2007
Altamura et al 2007
Audoin et al 2005
Axmacher et al 2007
Axmacher et al 2007
Axmacher et al 2007
Axmacher et al 2007
Axmacher et al 2009
Bedwell et al 2005
Bedwell et al 2005
Bedwell et al 2005
Beneventi et al 2007
Binder et al 2006
Binder et al 2006
Bunge et al 2001
Cader et al 2006
Cader et al 2006
Cairo et al 2004
Cairo et al 2004
Cairo et al 2004
Cairo et al 2004
Cairo et al 2004
Caldwell et al 2005
Callicott et al 1999
10
18
18
18
23
23
23
23
30
14
14
14
12
12
12
16
16
16
18
18
18
18
18
10
9
Contrast
Task>control
Task>control
activation modulated by load
Task>control
Task>baseline
Task>baseline
activation modulated by load
activation modulated by load
difficult>easy
Task>control
Task>control
Task>control
Task>baseline
Task>control
Task>control
difficult>easy
Task>control
activation modulated by load
Task>baseline
activation modulated by load
activation modulated by load
Task>baseline
activation modulated by load
Task>control
activation modulated by load
Task
Modality (visual)
Stimuli Type (letter)
n-back
Sternberg
Sternberg
Calculation
Sternberg
Sternberg
Sternberg
Sternberg
Sternberg
Sternberg
Sternberg
Sternberg
n-back
n-back
n-back
Sternberg
n-back
n-back
Sternberg
Sternberg
Sternberg
Sternberg
Sternberg
Sternberg
n-back
visual
visual
visual
auditory
visual
visual
visual
visual
visual
visual
visual
visual
visual
visual
visual
visual
visual
visual
visual
visual
visual
visual
visual
visual
visual
letter
letter
letter
numbers
faces
faces
faces
faces
figures
letter
letter
letter
shapes
letter
shapes
letter
letter
letter
letter
letter
letter
letter
letter
letter
numbers
Rottschy et al., Neuroimage 2012
Meta-analysis on working memory
Meta-analysis across 189 working memory experiments
→ Activations of all included experiments
How to quantify convergence?
Rottschy et al., Neuroimage 2012
Meta-analysis on working memory
Meta-analysis across 189 working memory experiments
ALE: Union of the modelled activation likelihood maps
Main effect
Convergence across all experiments investigating
the neuronal correlates of working memory
Rottschy et al., Neuroimage 2012
Meta-analysis on working memory
Sub-analyses
Examine the effects of:
§  Tasks / settings
§  Specific contrasts
or other experimental parameters
May allow inference on design effects
Do comparisons that are not possible in a single study
Verbal vs. non-verbal stimuli
More consistent activation for verbal stimuli
More consistent activation for non-verbal stimuli
Rottschy et al., Neuroimage 2012
n-back vs. Sternberg
Significantly higher convergence for activations in
n-back tasks vs. Sternberg tasks
Rottschy et al., Neuroimage 2012
Meta-­‐analysis on voli1onal movements -  PubMed search
Volitional movements:
Volition; self-initiated; self-paced; internally specified;
selection; choice; timing; etc.
movements ∩ fMRI; PET
Specific Inclusion-/Exclusion criteria
manipulation of a volitional aspect resulting in hand movements
à  53 papers including 103 experiments
Meta-­‐analysis on voli1onal movements -  PubMed search
Volitional movements:
Volition; self-initiated; self-paced; internally specified;
selection; choice; timing; etc.
movements ∩ fMRI; PET
Specific Inclusion-/Exclusion criteria
manipulation of a volitional aspect resulting in hand movements
à  53 papers including 103 experiments
Meta-­‐analysis on voli1onal movements Convergent ac1va1on across 103 studies of voli1onal hand movements Effects of movement (+ voli1on; 61exp.) ∩ Effects of voli1on (-­‐ movement; 42 exp.) Meta-­‐analysis on voli1onal movements Choice (59 exp.)
∩
Timing (39 exp.)
Meta-­‐analysis on voli1onal movements Task-set: choice (33 exp.)
PET; fMRI
block-design
event-related
fMRI
Preparation (19 exp.)
∩
∩
Task-set: timing (32 exp.)
Execution (17 exp.)
Meta-­‐analysis on voli1onal movements Meta-analyses allow to …
… base inference on sample sizes hardly to be reached
in individual studies
… identify those effects that are reproducible across
a larger number of publications
… integrate many variants of probing a particular
construct to test for effects of the experimental design
… relate different tasks or topics to each other
Aachen
Birgit Derntl
Kathrin Reetz
Göttingen
Oliver Gruber
Simon Eickhoff
Danilo Bzdok
Julia Camilleri
Edna Cieslik
Sarah Genon
Robert Langner
Claudia Rottschy
Veronika Müller
Rachel Pläschke
Andrew Reid
Deepthi Varikuti
Düsseldorf
Katrin Amunts
Alfons Schnitzler
Karl Zilles
Köln
Christian Grefkes
Jochen Michely
Leo Schilbach