Searching for systematic reviews:
problems, pitfalls and pragmatism
Alison Bethel
Morwenna Rogers
Information Specialists
ANATOMY OF A DATABASE
Database: A collection of records
RECORDS
(consist of
fields)
FIELDS
(individual
elements of
information)
TITLE
AUTHOR
SOURCE
SUBJECT
Feeding habits of orangutans of Borneo
Chegwin, D; Smith T
Journal of Primatology
Primates; great apes; feeding
TITLE
AUTHOR
SOURCE
SUBJECT
Tarsiers: the evolutionary link?
Branwell, P
Journal of Primatology
Primates; prosimians
TITLE
AUTHOR
SOURCE
SUBJECT
Japanese macaques and social groups
Smith, T
Journal of monkey psychology
Primates; monkeys; feeding
Searching: reasons to be fearful
•
•
•
•
•
Reliance on technology
Reliance on humans
Publishers & publications
A complicated process
Transparency: your errors will be seen!
Reasons to be fearful part 1:
We seem to have
encountered a
problem
Computer says no
•
•
•
•
•
•
•
•
Regular changes/upgrades to software
Different interfaces
Maximum number of downloads
Maximum number of search lines within strategy
Small number of records displayed
Non-consistent field codes, fields, syntax etc.
Automatic log outs
Making assumptions – what is it doing?
Human input
• Errors in manuscripts
• Errors transcribing to databases
• Misunderstanding of subject = poor
indexing
• Inadequate instruction prior to
search(e.g. inclusion/exclusion)
Publishers & publications
• Publishers in charge of databases: selective
content (no, you can’t just search PubMed!)
• Software compatibility issues
• Function not in the hands of the users
• Systematic review searching not a priority
Reasons to be fearful: search process
Reasons to be fearful: search process
•
•
•
•
•
•
Too many hits
Not enough hits
Missing records
Numbers don’t add up
Too many databases! Too much choice.
When do you stop?
Be prepared!
• You have a broad understanding of how much is
out there
• You have a protocol
• You have a PICO or similar
• You are familiar with the resources
Scoping searches – why??
• Has it been done before?
• How was it done?
• What was the result?
• Size of the evidence base
• Related work (background reading)
• Search terms
• Key papers
Scoping searches – some sources
Master search strategy
• Use your PICO
• Identify synonyms
Team
Wikipedia!
Key references
• Use OvidSp if you can
Syntax
Save searches
Edit searches
Search strategy transfer
Search term table
Effectiveness of mealtime interventions for elderly
residents in care homes on nutritional outcomes: A
systematic review
Population
(Group 1)
OR
Population
(Group 2)
Intervention
(Group 3)
Comparator
(Group 4)
Outcomes
(Group 5)
“Old* people” “Care adj2 home*”
Mealtime*
Weight
Elderly
“Residential home”
Meals
Nutrition*
Geriatric*
“Nursing home”
Dining
Resident?
“Long term care”
Dinner
Breakfast
Lunch
AND
Example of a search strategy
Database: Ovid MEDLINE(R) In-Process & Other Non-Indexed Citations and Ovid MEDLINE(R) <1946 to Present>
-------------------------------------------------------------------------------1 meta-analysis.pt. (34938)
2 meta?analy*.ti,ab. (1149)
3 meta analy*.ti,ab. (44285)
4 (systematic adj3 review*).ti,ab. (38850)
5 (methodologic* adj3 review*).ti,ab. (1640)
6 (methodologic* adj3 overview*).ti,ab. (127)
7 review*.ti. (227326)
8 1 or 2 or 3 or 4 or 5 or 6 or 7 (282830)
9 meal*.ti,ab. (46556)
10 nutrition*.ti,ab. (157255)
11 food.ti,ab. (216127)
12 eating.ti,ab. (38571)
13 dining.ti,ab. (581)
14 feeding.ti,ab. (121833)
15 9 or 10 or 11 or 12 or 13 or 14 (496579)
16 elderly.ti,ab. (152718)
17 (old* adj (people or resident*)).ti,ab. (15408)
18 resident*.ti,ab. (98742)
19 16 or 17 or 18 (256123)
20 (care adj (setting* or home* or unit* or residence)).ti,ab. (89197)
21 (residential adj (care or unit* or home*)).ti,ab. (2373)
22 nursing home*.ti,ab. (19978)
23 20 or 21 or 22 (109793)
24 8 and 15 and 19 and 23 (27)
Filters
• What are they
• When to use them
Databases: avoiding the pitfalls
• Extra time
• On-line help
• Know your syntax, fields
codes etc.
• Use field codes
• Set up accounts
• Save your search strategy
• Differences between databases
Your search: cover all bases!
• Subject headings AND free text
• Proofread
• Supplementary searches
–
–
–
–
forwards citation chasing
backwards citation chasing
hand-searching key journals
speak to experts
BUT
• Be realistic
When do you stop?
• Use gold standard of key references
• Aggressively seek potentially non-captured
studies
• Look at results from similar reviews
• Search sources likely to yield greatest results (law
of diminishing returns) and establish a cut-off
point
• Theoretical saturation (for qualitative searching)
Adapted from Booth 2010
Reasons to
be cheerful
• Focused plan
• Advance knowledge of databases and hosts
• Carefully structured search strategy approved
by whole team
• Plenty of time allowed for search
• Know when to stop
• Be realistic
• EST search and review clinics
References
Booth A: How much searching is enough? Comprehensive versus optimal
retrieval for technology assessments.
Int J Technol Assess Health Care 2010, 26(4):431-5.
Stevinson C, Lawlor D. Searching multiple databases for systematic reviews:
Added value or diminishing returns?
Complement Ther Med. 2004;12:228-232.
Thank you for listening
Contact us:
[email protected]
[email protected]
Twitter: @EvidSynthTeam