Evaluating the Medical Literature Clista Clanton, MSLS, AHIP January 2009 Start at the top and work your way down. Haynes, RB. ACPJC Nov/Dec 2006;145(3):A8-9 Computer system Clinical Evidence or PIER (UpToDate) ACP Journal Club, InfoPOEMS, Dynamed Cochrane Library, PubMED Clinical Queries, BMJUpdates, guidelines Original Studies If an original study is your only option……. Original Studies IMRAD format  Introduction: why the authors decided to conduct the research.  Methods: how they conducted the research and analyzed their results.  Results: what was found. And  Discussion: what the authors think the results mean. PP-ICONS        Problem Patient or population Intervention Comparison Outcome Number of subjects Statistics Flaherty, Robert J. A simple method for evaluating the clinical literature. Fam Prac Mgt, May 2004;47-52. Available online at http://www.aafp.org/fpm/20040500/47asim.html. Scenario  You just saw a nine-year old patient with common warts on her hands. She is an ideal candidate for cryotherapy. Her mother has heard about treating warts with duct tape and wants to know if you would recommend this treatment. Clinical Question  What is your clinical question?  PICO: Patient, Intervention/Comparison, Outcome  Write your question down, it can help you to clarify exactly what you are looking for. “Is duct tape an effective treatment for warts in children?” Search  After you have your clinical question, search the appropriate databases:  Dynamed, PIER, UpToDate, Cochrane, Clinical Evidence, Essential Evidence Plus  PubMed  Focht DR 3rd, Spicer C, Fairchok MP. The efficacy of duct tape vs cryotherapy in the treatment of verruca vulgaris (the common wart). Arch Pediatr Adolesc Med. 2002 Oct;156(10):971-4.  Abstract OBJECTIVE: To determine if application of duct tape is as effective as cryotherapy in the treatment of common warts. DESIGN: A prospective, randomized controlled trial with 2 treatment arms for warts in children. SETTING: The general pediatric and adolescent clinics at a military medical center. PATIENTS: A total of 61 patients (age range, 3-22 years) were enrolled in the study from October 31, 2000, to July 25, 2001; 51 patients completed the study and were available for analysis. INTERVENTION: Patients were randomized using computer-generated codes to receive either cryotherapy (liquid nitrogen applied to each wart for 10 seconds every 2-3 weeks) for a maximum of 6 treatments or duct tape occlusion (applied directly to the wart) for a maximum of 2 months. Patients had their warts measured at baseline and with return visits. MAIN OUTCOME MEASURE: Complete resolution of the wart being studied.  RESULTS: Of the 51 patients completing the study, 26 (51%) were treated with duct tape, and 25 (49%) were treated with cryotherapy. Twenty-two patients (85%) in the duct tape arm vs 15 patients (60%) enrolled in the cryotherapy arm had complete resolution of their warts (P =.05 by chi(2) analysis). The majority of warts that responded to either therapy did so within the first month of treatment. CONCLUSION: Duct tape occlusion therapy was significantly more effective than cryotherapy for treatment of the common wart. Problem (PP-ICONS)  What is the clinical condition that was studied in the article? OBJECTIVE: To determine if application of duct tape is as effective as cryotherapy in the treatment of common warts.  The problem studied should be sufficiently similar to your clinical problem, or the results will not be relevant. Patient or Population (PPICONS)  Is the study group similar to your patient or practice? SETTING: The general pediatric and adolescent clinics at a military medical center. PATIENTS: A total of 61 patients (age range, 3-22 years)  If the patients in the study are not similar to your patient (older, sicker, different gender or more clinically complicated), the results may not be relevant. Intervention (PP-ICONS)  Is the intervention the same as what you are looking for?  Could be a diagnostic test or a treatment The patient’s mother has heard about treating warts with duct tape and wants to know if you would recommend this treatment. Comparison (PP-ICONS)  The comparison is what the treatment is tested against.  Could be a different diagnostic test, another therapy, placebo, or no treatment at all. INTERVENTION: Patients were randomized using computer-generated codes to receive either cryotherapy (liquid nitrogen applied to each wart for 10 seconds every 23 weeks) for a maximum of 6 treatments or duct tape occlusion (applied directly to the wart) for a maximum of 2 months. Outcome (PP-ICONS)  Disease-oriented outcomes (DOEs): usually reflect changes in physiologic parameters.  It has long been assumed that improving the physiologic parameters of a disease will result in a better outcome, but this is not always true.  Patient-oriented evidence that matters (POEMs): look at outcomes such as morbidity, mortality and cost.  Therefore, DOEs are interesting but of questionable relevance, whereas POEMs are very interesting and very relevant. MAIN OUTCOME MEASURE: Complete resolution of the wart being studied. Number (PP-ICONS)  Number of subjects in the study is crucial in whether accurate statistics can be generated from the data.  Too few patients may not be enough to show that a difference really exists between intervention and comparison groups (power of a study).  Many studies contain <100 subjects, which is usually inadequate to provide reliable statistics.  Good rule of thumb – 400 subjects needed.8 51 patients completed the study Krejcie RV, Morgan DW. Determining sample size for research activities. Educational and Psychological Measurements. 1970;30:607-610. Statistics (PP-ICONS)  Relative risk reduction (RRR): the percent reduction in events in the treated group compared to the control group event rate.  Not a good way to compare outcomes  Amplifies small differences and makes insignificant findings appear significant  Doesn’t reflect the baseline risk of the outcome event  Can make weak results look good, therefore  Popular and will be reported in almost every journal article  Ignore – it can mislead you RRR would be (85 percent – 60 percent/60 percent x 100 = 42 percent I.e. 42 percent more effective than cryotherapy in treating warts Statistics (PP-ICONS)  Absolute risk reduction (ARR): the difference in the outcome event rate between the control group and the experimental group. ARR for the wart study is the outcome event rate (complete resolution of warts) for duct tape (85 percent) minus the outcome event rate for cryotherapy (60 percent) = 25 percent  A better statistic to evaluate outcome, as it does not amplify small differences, but shows the true difference between the experimental and control interventions. Statistics (PP-ICONS)  Number needed to treat (NNT): number of patients who must be treated to prevent one adverse outcome OR the number of patients who must be treated for one patient to benefit  Single most clinically useful statistic  Easy to calculate, simply the inverse of the ARR. For the wart study, the NNT is 1/25 percent = 1/0.25 = 4 4 patients need to be treated with duct tape for one to benefit more than if treated by cryotherapy  The lower the NNT, the better. For primary therapies, an NNT of 10 or less is good, with less than 5 being very good.  For preventive interventions, the NNT will be higher. An NNT for prevention of less than 20 might be particularly good. Intention to Treat Analysis  Attrition: Were patients lost to follow-up, and if so, why? Intention to treat: subjects are analyzed according to the categories into which they were originally randomized. – Benefits of a treatment are more difficult to demonstrate with intention-to-treat analysis. – Helps to mitigate differences by including subjects who are unlikely to have experienced benefit from the intervention. Six patients from cryotherapy group and 4 patients from the duct tape group were lost to follow-up (16% of patients). Worst case scenario: 6 cryotherapy patients had wart resolution and the 4 duct tape patients had residual wart. Wart resolution would then be: duct tape 78% and cryotherapy 68% (95% CI, -17 to 28) – therefore not a statistically significant difference between the two treatments. Christakis DA, Lehmann HP. Is duct tape occlusion therapy as effective as cryotherapy for the treatment of the common wart? Arch Pediatr Adolesc Med, Oct 2002; vol. 156; 975-977. Best Type of Study for Your Question Type of Question Suggested Best Type of Study Therapy RCT > cohort > case control > case series Diagnosis Prospective, blind comparison to gold standard Etiology / Harm RCT > cohort > case control > case series Prognosis Cohort study > case control > case series Prevention RCT > cohort study > case control > case series Clinical Exam Prospective, blind comparison to gold standard Cost Economic analysis Questions of therapy, etiology and prevention which can best be answered by RCT can also be answered by a meta-analysis or systematic review. Resources  Presentation/handouts available at: http://southmed.usouthal.edu/library/ebmclass/rotationswinterspring.htm Assignment      Identify a clinical problem with a patient Formulate a clinical question using PICO Search the literature for appropriate article(s) Evaluate the article(s) Complete the online assignment within two weeks after date of lecture.  http://southmed.usouthal.edu/library/ebmclass/rotationswinterspring.htm