Clinical practice guidelines for acute diarrhea
in children: Methodological quality
Iván D. Flórez1. Javier Contreras1 Javier Sierra1,
María E. Tamayo1, Luz H. Lugo1, Claudia Granados2, Germán
Briceño3, Juan M. Lozano4, Jorge L. Acosta5
1- Universidad de Antioquia; Medellín, Colombia
2. Pontificia Universidad Javeriana; Bogotá, Colombia
3. Fundación Cardio-infantil; Bogotá, Colombia
4. University of Florida, Miami, USA
5. Universidad del Norte, Barranquilla, Colombia
Background
•  Acute diarrhea (AD) is the second most common disease in
children
•  AD is globally associated with high morbidity in developed
countries and with high mortality in undeveloped countries.
•  The Colombian Ministry of Health financed the development
CPG of some diseases in order to improve health care system
and to do cost-effectiveness analysis. Diarrhea in children was
one of them
•  We formed a guideline developer group (GDG) in order to
develop the Clinical Practice Guidelines (CPG) of prevention,
diagnosis and treatment of diarrheal diseases in children
Background
•  Before developing a de novo-CPG it is
recommended to assess published guidelines in
order to consider adaptation of a high quality CPG
•  There were a lot of CPG of diarrhea and
gastroenteritis in children available in the literature
•  The quality of published guidelines on childhood
diarrhea was assessed by Lo Veccio et al. in 2011,
but it had some flaws
Evaluation of the Quality of Guidelines for A
Gastroenteritis in Children With the AGREE Ins
Evaluation of the Quality of Guidelines for Acute
Gastroenteritis in Children With the AGREE Instrument
!
Andrea Lo Vecchio, !Antonietta Giannattasio, yChristopher Duggan, zSalvatore De Masi,
§
Maria Teresa Ortisi, jjLuciana Parola, and !Alfredo Guarino
ABSTRACT
Problems:
Aim: The aim of the study was to assess the quality of clinical practice
guidelines (CPGs) using the Appraisal of Guidelines for Research and
Evaluation (AGREE) instrument, a validated international tool.
Materials and Methods: CPGs were identified by searching MEDLINE
(1966 – January 2009) and Embase (1988 – January 2009), CPG databases,
and relevant Web sites of agencies and organizations that produce and/or
endorse guidelines. Included in the study were CPGs in English that
addressed the management of acute gastroenteritis in children. Retrieved
CPGs were evaluated with the AGREE instrument for quality assessment
by 6 independent reviewers. AGREE consists of 6 domains for a total of
23 items.
Results: Nine CPGs were identified. Four were evidence based (EB) and
2 of these included tables of evidence. Eight CPGs (88%) scored <50% for
‘‘applicability,’’ 7 (77%) for ‘‘stakeholder involvement,’’ and 6 (66%) for
‘‘editorial independence.’’ Compared with non-EB CPGs, EB CPGs had
higher quality scores for all AGREE domains, with a better score for ‘‘rigor
of development’’ (P < 0.001), ‘‘stakeholder involvement’’ and ‘‘clarity of
presentation’’ (P < 0.01), and applicability (P < 0.05). Over time, the
quality of guidelines tended to improve. The main recommendations of
CPGs were similar. However, there were differences in the treatment of
diarrhea, namely based on the settings and circumstances in which CPGs
were produced.
• 
• 
• 
• 
• 
Conclusions: The overall quality of CPGs on acute gastroenteritis
management in children is fair. Aims, target population, synthesis of
evidence, formulation of recommendations, and clarity of presentation
are points of strength. Weak issues are applicability, including identification of organizational barriers and adherence parameters, and cost/efficacy
analysis.
Andrea Lo Vecchio, !Antonietta Giannattasio, yChristopher Duggan, zSalvatore
§
jj
First version of the AGREE
instrument
Maria Teresa Ortisi, Luciana Parola, and !Alfredo Guarino
!
Key Words: acute gastroenteritis, AGREE instrument, child, guidelines
Search was limited to English language CPG
C
Databases and five web-sites of CPG or associations
Date search
limit: 2008
ABSTRACT
In 2009 were published: 1 guideline in Colombia
Conclusions: The overall quality of CPGs
Aim:
The
aim
of
the
study
was
to
assess
the
quality
of
clinical
practice
and 2 guidelines from NICE and ESPGHAN
management in children is fair. Aims, target
guidelines (CPGs) using the Appraisal of Guidelines for Research and
evidence,
formulation of recommendations, and
•  In 2009, were
published
some
Cochrane
Systematic
Evaluation (AGREE)
instrument, a validated
international
tool.
are points of strength. Weak issues are applicabi
reviews about
in bydiarrhoea
Materials andtreatments
Methods: CPGs were identified
searching MEDLINE
tion of organizational barriers and adherence para
(1966–January 2009) non-evidence
and Embase (1988–January 2009), CPG
databases, CPG
•  Authors included
based
analysis.
and relevant Web sites of agencies and organizations that produce and/or
•  This paper was not enough for us to do an assessment
endorse guidelines. Included in the study were CPGs in English that Key Words: acute gastroenteritis, AGREE instru
of the quality
theses
addressed theof
management
of acute CPG
gastroenteritis in children. Retrieved
CPGs were evaluated with the AGREE instrument for quality assessment (JPGN 2011;52: 183–189)
Received February 12, 2010; accepted April 2, 2010.
From the !Department of Pediatrics University of Naples ‘‘Federico II,’’
Naples, Italy, the yDivision of Gastroenterology and Nutrition, Children’s Hospital Boston, and Department of Pediatrics, Harvard Medical
School, Boston, MA, the zDepartment of Epidemiology and Guidelines
of the Italian Institute of Health (Istituto Superiore di Sanità), Rome,
Italy, the §Hospital of Sant’Anna of Como, Italy and member of the
Accreditation and Quality Improvement Working Group of Italian
Society of Pediatrics, and the jjHospital of Magenta, Azienda Ospedaliera ‘‘Ospedale Civile di Legnano’’ Italy and member of the Accreditation and Quality Improvement Working Group of the Italian Society of
Pediatrics.
Address correspondence and reprint requests to Prof Alfredo Guarino,
Department of Pediatrics University of Naples ‘‘Federico II,’’ Via
Pansini 5, 80131 Naples, Italy (e-mail: [email protected]).
Two of the authors have been involved in the production of 2 guidelines that
were included in this study (A.G. and C.D.). Neither guideline ranked
first in the AGREE evaluation, and in addition, the concordance between
raters was good, suggesting that there were no biases in the evaluation. In
addition, A.G. promoted and coordinated the present study but did not
take part in the guideline assessment as evaluator.
The authors report no conflicts of interest. There is no potential conflict of
interest in any part of the study and there was no form of payment given
to anyone to produce the manuscript.
Copyright # 2011 by European Society for Pediatric Gastroenterology,
Hepatology, and Nutrition and North American Society for Pediatric
Gastroenterology, Hepatology, and Nutrition
DOI: 10.1097/MPG.0b013e3181e233ac
JPGN
"
Volume 52, Number 2, February 2011
(JPGN 2011;52: 183 – 189)
linical practice guidelines (CPGs) are systematically developed statements to assist practitioners in making decisions
about appropriate health care in specific clinical circumstances (1).
Their purpose is to make explicit recommendations with a definite
intent to influence what clinicians do. The primary goal of CPGs in
pediatrics is to improve the health of infants and children by
ensuring that they receive up-to-date, evidence-based (EB) care.
CPGs are a major tool to improve the quality of care. Several studies
have shown that adherence to EB guidelines leads to improvement
in the quality of care provided (2,3). For many health conditions,
there is a gap between what medical science has shown to be
effective practice and what is actually done (4).
The number of CPGs is rapidly mounting also in pediatrics.
However, the plethora of CPGs has been accompanied by growing
concern about differences among guideline recommendations and
about the quality of guidelines (5 – 8). How does one define the
quality of guidelines? A ‘‘good’’ guideline should be scientifically
valid, usable, and reliable, and should improve the outcome of
patients; however, it is rarely known how a guideline performs in
clinical practice. Evaluation of CPGs should include both methods
used to develop recommendations and applicability of recommendations (benefits, adverse effects, and costs).
An international group of researchers, the Appraisal of
Guidelines for Research and Evaluation (AGREE) Collaboration,
developed and validated a specific instrument to assess the quality
of CPGs based on theoretical assumptions (9). A recent assessment
of the quality of pediatric guidelines with the AGREE instrument
demonstrated better results for pediatric than for adult CPGs (10).
The best performers were CPGs published and endorsed by the
American Academy of Pediatrics (AAP) or registered in the
National Guidelines Clearinghouse (NGC).
Acute gastroenteritis (AGE) remains a common cause of
morbidity and mortality among infants and children worldwide. In
industrialized countries, the disease is relatively mild and generally
self-limiting, but nevertheless can have a major effect on the quality
of life of infected children and their families. AGE is a major
cause of outpatient visits and hospital admissions in developed
countries, and consequently it has a substantial effect on health
costs. Several guidelines for the management of AGE in children
are available. However, only a minority of physicians fully comply
183
Objective
•  To assess the quality of CPG on acute
diarrhea or gastroenteritis in children using
the AGREE-II instrument (Spanish version).
Methods
•  Systematic review of CPG
•  Databases:
o  Electronic databases: EMBASE, MEDLINE, LILACS
o  National clearinghouses
o  Non-electronic sources: Hand-searching:
•  Known CPG by authors, researchers, members of the
group and clinical experts
•  Pediatrics, gastroenterology, familiar medicine and
nursing congresses and textbooks
•  We searched from June to August of 2011
Methods
•  We exclude guidelines that:
o  Were not Evidence-based
o  Had a different scope
o  Exclusive adult population
o  Diarrhea in children with chronic diseases
•  Each EB-guideline was independently assessed using
AGREE-II (Spanish version) by three (3) clinical
epidemiologists
•  Appraisers (MSc in Clinical epidemiology)
o  Genera practicioner (1), Pediatrician (6)
Methods
•  The assesment was made based on the information
presented in the published version of the CPG and
on the web site of the organization, institution or
journals
•  We acceded when possible to evidence tables and
search strategies by contacting authors and
institution by mail. Methods
•  Spanish-AGREE II instrument has 23 items grouped in
6 domains and 2 final items for the global evaluation.
•  The score is calculated with proportions for each
domain
•  There is not a minimum score to recommend a CPG
•  Rigor of development was chosen as the most
important domain based on Colombian guidelines
methodology manual
•  60% was the cut off point to consider a high quality
guideline
Methods
•  The score of each domain was calculated based in
the 3 appraisers evaluation
•  Differences between the scores in each items were
analyzed by one of the appraisers.
•  When differences between appraisers were more
than 4 points, they were asked to revaluate their
score, in order to get a consensus
•  We calculated medians and interquartile ranges
(IQR) of the scores in each domain
APPRAISAL OF GUIDELINES
fOR RESEARCH & EVALUATION II
INSTRUMENT
___________________________________________________________________________________
INSTRUMENTO AGREE II
The AGREE Next Steps Consortium
May 2009
INSTRUMENTO PARA
LA EVALUACIÓN
DE GUÍAS DE PRÁCTICA CLÍNICA
Results
•  We found 63 diarrhea guidelines
•  17 of which (26.9%) were evidence-based.
•  The rest (46 CPG) were: protocols, narrative reviews, Non
evidence –based guidelines, experts consensus, guidelines of
associations and organizations (WHO), cost-effectiveness
studies
•  10 in English language: Canada, India, UK, USA, Europe,
Australia
•  7 in Spanish language: Colombia, México, Guatemala, Costa
Rica, Spain
•  Peer-reviewed journals-databases (6), textbooks (2), guideline
developer web-site (1), academic institutions or scientific
association (4) or government organizations (4)
Results
•  Domains 1 (scope and purpose) and 4 (clarity of
presentation) had the highest scores: 74 (60-94%)
and 78 (56,5-91%), respectively
•  Domains 5 (applicability) and 6 (editorial
independence), had the lowest: 13 (2-24,5%) and 8
(0-58%), respectively
•  Only five guidelines scored over 60% in the third
domain (rigour of development) and were
considered as “recommended”.
Results
Recommended (5)
Not recommended (12)
•  Domain 5 : 31 (6-69%)
•  Domain 5 : 10 (0-15%)
•  Domain 6: 54 (0-97%)
•  Domain 6: 8 (8-13%)
•  Domain 1: 93,5 (89-100%)
•  Domain 1: 63 (17-79%)
•  Domain 4: 93,5 (89-95%)
•  Domain 4: 69 (57-80%)
•  Domain 3: 70 (60-42%)
•  Domain 3: 27 (20-40%)
•  Domain 2: 65 (42-95%)
•  Domain 2: 26 (18-32%)
Discussion
•  There is an urgent need to improve the quality of
CPGs in acute diarrhea in children by incorporating
high quality standards in the development process
based on the key points of a high-quality CPG
•  Most of CPG were from developed countries
•  There are several diarrhea CPG available in Spanish
language but its quality is quite low in comparison
to English languages ones
•  Two CPG were retrieved form Spanish textbooks
Discussion
•  There are a lot of CPG that are called EB-guidelines
and they are not.
•  We retrieved CPG from databases, textbooks,
developers websites, national clearinghouses. The
search of CPG must be much wider than that for
primary studies
•  Diarrhea is a common disease in children. It is
desirable to have the best evidence-based
recommendation available.
Discussion
•  First (1th) and 4th domains scored high (upper 60%) in
both recommended and not recommended CPG
•  Fifth and 6th domains scored low in both recommended
and not recommended CPG
•  Third domain must be the one that makes the difference
•  Developers are concerned about describing Scope and
objectives and about the clarity of presentation, instead
of being rigorous in methodology
•  We must be alert about the 3rd domain better than
others that seem “attractive”: Clarity of
recommendations and scope
Limitations
•  We have conclusions that could not be reproducible
in other diseases
•  We had a low number of CPG
•  The 60% as a cut-point was recommended by the
Colombian guidelines, and the group agreed with
this score.
•  What about another cut-points? 70 – 75 – 80 % ?
•  AGREE collaboration doesn’t recommend a score,
so it could be controversial our cut-off point
Conclusions
•  There is low quality in Spanish language CPG on
diarrhea
•  Most of CPG on diarrhea were not evidence based
•  Most of evidence based CPG had low to moderate
quality
•  In AGREE-II, Third domain makes the difference
•  Scope and purpose, and clarity could be good
even in low quality guidelines
Danke!
ivoﬂ[email protected]