Evidence-‐Based Medicine 4 April, 2013 Workbook Formulate a clinical question (PICO) Search for the best available evidence Page Contents 2 Formulating a clinical question (PICO) PICO practice abstracts Searching guide Randomized Clinical Trial critical appraisal 3 – 8 9 10 Appraise the evidence: Study design Results 11 Statistical interpretation Apply the evidence to the individual patient Įrodymais pagrįstos medicinos draugija www.medevidence.eu www.cebm.net 1 Vilniaus Universiteto Medicinos Fakultetas Formulating a Clinically Relevant Question Prior to the literature search it is most convenient to begin with formulating a focused clinical question that would lead you to precise answers. Systemic formulation of thoughtful question would provide you with key words to ease your literature search. Furthermore, formulating a good clinical questions would uncover what kind of research evidence may answer your query. 4 major sections are identified in formulation of an answerable clinical question (PICO): P I C O Example Patient/Population/Problem The group of patients I am concerned about Intervention What intervention we are considering? Comparison What other intervention I am comparing to? Outcome What effect of the intervention am I seeking for? Men with metastatic prostate cancer Abiraterone Compared to placebo Longer survival or no disease progression Question: In men with metastatic prostate cancer would abiraterone prolong survival? Our question whether abiraterone would be effective against metastatic prostate cancer may be best answered by Randomized Clinical Trial. Our search ‘metastatic prostate cancer abiraterone survival’ in PubMed gives three key hits (for moer on how to perform the literature search, please refer below): Abiraterone acetate for treatment of metastatic castration-‐resistant prostate cancer: final overall survival analysis of the COU-‐AA-‐301 randomised, double-‐blind, placebo-‐controlled phase 3 study (Fizazi, Lancet Oncol. 2012) Abiraterone in metastatic prostate cancer without previous chemotherapy (Ryan, NEJM 2013) Abiraterone and increased survival in metastatic prostate cancer (De Bono, NEJM 2012) 2 P I C O Question: 3 P I C O Question: 4 P I C O Question: 5 P I C O Question: 6 P I C O Question: 7 Formulate your own question and conduct search: P I C O Question: 8 Quick guide to searching: First formulate a question using PICO Searching tips: • • • • • • Think about different key words & phrases to describe your search terms Combine searches using OR, AND, *: o OR – broadens the search, used to combine synonyms e.g. “common cold” OR cough o AND – focuses the search, used to combine different concepts e.g. “common cold AND “vitamin c” o * -‐ truncation, use for words with multiple endings e.g. chil* will search for child, children, childhood. Use ( ) to group search terms e.g. (“common cold” OR cough) AND “vitamin c” Use limits if available e.g. language, publication date, study type Use “ ” to search for phrases e.g. “common cold” Use the MeSH Database to conduct a subject search – listed under More Resources on the PubMed home page (see PubMed tutorial for more help) Useful resources for getting started: • • • • Cochrane Library – http://www.thecochranelibrary.com Pubmed – http://www.pubmed.gov UpToDate – http://www.uptodate.com TRIP database -‐ http://www.tripdatabase.com/ More help? • • • • Finding the Evidence – Videos – http://www.cebm.net/index.aspx?o=1038 PubMed tutorials – http://www.nlm.nih.gov/bsd/disted/pubmed.html Cochrane tutorials – http://www.thecochranelibrary.com/view/0/HowtoUse.html Searching Skills Toolkit by Caroline De Brun and Nicola Pearce-‐Smith Example search: Clinical question: Is acupuncture effective as a smoking cessation technique? P = smokers I = acupuncture C = N/A O = cessation Search terms and synonyms: P = smoker, smokers, smoking, tobacco I = acupuncture, acupressure, chinese medicine 9 O = cessation, stop, quit Simple search on PubMed: Go to PubMed http://www.pubmed.gov Similarly other sites could be tried. Within PubMed either in Clinical Queries: From the PubMed home page – click on Clinical Queries (listed under PubMed Tools) Type your search terms in the box e.g. (smoker* OR smoking OR tobacco) AND (acupuncture OR acupressure OR “chinese medicine”) View results for systemic reviews & therapy (narrow) Or in the main search screeen (using advanced search) enter the following as separate searches: #1  smoking OR smoker* OR tobacco #2  acupuncture OR acupressure OR “chinese medicine” #3  cessation OR quit* OR stop* Then, in the last search combine the search results: #4  #1 AND #2 AND #3 From the results display page, refine search by article type e.g. meta-‐analysis or randomized controlled trial 10 Critical appraisal of the evidence (Randomized Controlled Trials) Reliable study results require a rigorous study design to ensure good control. Although finding the best available evidence is crucial in medical practice, appraisal of literature is key in weighing the reliability and clinical importance of the results. There are a few key points in analysis of randomized studies that are important to consider in assessing the validity of study’s outcomes and conclusions. 1. Randomization of patients Were patients randomized into treatment and control arms? Centralized computer randomization is ideal and used in most multi -‐ centered trials. Usually Methods section mentions how patients were allocated to different research groups. 2. Similarity of study groups at the start of the trial If randomization was performed properly, research groups should be similar regarding all the possible confounding variables. The more similar the treatment and control groups prior to the treatment – the better it is. The Results section should contain the information concerning ‘Baseline Characteristics’ of the randomized groups on a number of variables that could affect the outcome. 3. Equal treatment of the groups Apart from the intervention itself both groups should be treated in the same way, otherwise: ie. all other treatments and tests should identical in all of the experimental groups. Methods section should name all other permitted additional treatments to experimental groups and Results section the actual use of them. 4. Were all the patients who entered the trial accounted for? Are the losses of patients in both groups similar and what accounts for the loss of patients? Losses to follow-‐up should be minimal – preferably less than 20%. However, if patients have the outcome interest, then even small losses to follow-‐up can bias the results. Patients should be analyzed in the groups to which they were assigned. Results section should say how many patients were randomized and how many were actually included in the analysis. Reasons for dropout are usually outlined in the results section. 5. Were the treatment methods masked? Was the study blinded? It is ideal for study to be ‘double-‐blinded’ – that is both patients and investigators are unaware of treatment allocation. If the outcome is subjective (eg. Symptoms or function) then blinding is absolutely critical. If the outcome is objective (eg. death) then blinding is less critical. Methods section should both mention whether the treatments were masked and whether the assessors were aware of patient’s treatment. 6. What are the results? How large is the treatment effect and how precise was the estimate of the treatment effect? Understanding of several commonly used statistical values is important. View the section below. 11 Statistical Interpretation Relative Risk (RR) Describes the risk of outcome in the treatment group relative to the control group. RR is a ratio of two risks. For example, if 30% of people who were given antibiotics had a sore throat 3 days after and 60% of those without antibiotics had a sore throat, then RR=30%/60%=0.5 A RR=1 (null value) means that there is no effect of treatment and there is no difference between treatment and control groups. RR>1.0 means that the effect is generally harmful, while RR<1.0 means that the treatment is protective. The more the RR is away from the null value – the greater is the protective effect. Absolute Risk Reduction (ARR) It is the difference between the event outcome rate in control and treatment groups. ARR=rate of outcome in the control group – rate of outcome in the treatment group In the sore throat example antibiotic treatment had 60%-‐30%=30% absolute risk reduction. An ARR of zero indicated that there is no effect of treatment. ARR is clinically informative as it takes into account the prevalence of the outcome. Relative risk reduction (RRR) Tells us what is the effect of the treatment in reducing the incidence of outcome as a difference. RRR=ARR/risk in the control group Or RRR=1-‐RR From the sore throat example above: RRR=30%/60%=50% Or RRR=1-‐0.5=0.5=50% Use of antibiotics reduced sore throat incidence after three days by 50%. Relative risk reduction is the most commonly reported measure of treatment effects. Number Needed to Treat (NNT) Tells us how many patients we need to treat with experiment therapy to prevent one bad outcome. When weighted with side effects of treatment, NNT can give some clinical significance. NNT=1/ARR In our example NNT=1/0.3=3.33. We would need to treat just over three people with antibiotics to prevent permanent sore throat. Note that NNT is a significant measure only when the study population is similar to the target population to whom the treatment is intended. Outcome and Exposure Status Outcome present Outcome not present Exposed to treatment: treatment group Not exposed to treatment: control group A B Number exposed (A+B) C D Number not exposed (C+D) Number with outcome (A+C) Number without outcome (B+D) 12 Odds Ratio (OR) The odds ratio is the ratio of the odds of a disease occurring in the presence of an exposure relative to the odds of the event occurring in the absence of the exposure. The odds ratio is commonly used in case-‐ control studies. From the outcome and exposure table, the odds ratio can be defined by the following formula: OR=odds of disease in the presence of exposure/odds of disease in the absence of exposure =A/B divided by C/D = AD/BC Therefore, this is the odds of the outcome of the treatment group relative to the odds of the outcome in controls. OR=1.0 means that there is no difference between the treatment and control groups. OR>1.0 for an adverse outcome means an increased risk. Hazard Ratio (HR) HR is used in survival analysis, this is a probability of a hazard at time ‘t’ in the treatment group compared with the probability of a hazard at time ‘t’ in the control group. For example if people in the treatment group are dying at twice the rate per unit time compared with the control group, then HR would be 2.0. 95% Confidence interval (CI) The terms outlined above (RR, OR, HR) are all estimates of the effect of the study factor on the population in question. However, there will always be some error associated with this, particularly if there are only a small number of people taking part in the study (small sample size). The 95% CI is the range within which we are 95% confident that the true estimate of the effect lies. Note, that if the 95% CI for an OR or RR value crosses 1.0 (the point of no effect), then it is possible that the true effect is none – the effect is not statistically significant. P-‐value The P-‐value is a number between 0 and 1.0 and is used to demonstrate the strength of conclusion drawn from clinical trial data. A p-‐value is the probability of the observed difference being due to chance. Traditionally, if p-‐value is <0.05, the result is considered statistically significant and treatment effect is not explained by chance. Very small P-‐values in clinical trials can be achieved either by obvious difference between treatment and control groups or by having a small difference but huge sample size that allows reaching statistical significance. Beware that P-‐values reflect only the statistical significance, but not clinical significance and result may be statistically significant without tangible clinical significance. 13 Forest Plot Comparison: Warfarin vs. Placebo in neonates Outcome: Removal of peripheral venous catheter due to complications A systematic review provides a summary of the data from the results of a number of individual studies. If the results of the individual studies are similar, a statistical method (meta-‐analysis) is used to combine the results from the individual studiesaccording to their size. The individual results of the studies need to be expressed in a standard way, such as a relative risk or odds ratio. Results are usually displayed in a figure like one above called a forest plot. The forest plot above represents a meta-‐analysis of 4 trials that individually assessed the effects of warfarin treatment in neonates on forced removal of venous catheter becasue of complications. Individual studies are represented by a blue square and horizontal line, which corresponds to the point estimate and 95% confidence interval of the relative risk. The size of the blue square reflects the weight of the study in the meta-‐analysis. The vertical line at 1.0 represnts ‚no effect‘ value. When the confidence interval of the study overlaps 1 it indicates that the result is not signifcant at conventional levels (P>0.05). The diamond at the bottom represents the combined or pooled relative risk of all 4 trials with its 95% confidence interval. In the case above it shows that treatment does not have any effect on catheter patency (RR=1.00 95% CI 0.85-‐1.16). P=0.96 and meta-‐analysis demonstrates that neither clinical nor statistical significance can be achieved with meta-‐analysis using trials done so far and no recommendations on warfarin use can be made. Kaplan – Meier survival curves 14 They depict the survival of patient population in treatment and non-‐treatment arms. As the effect of treatment may be uncovered earlier than all events occur, Kaplan – Meier curver is a useful systemic summary of treatment effect that can be generated even in the real time of the study. Curve above is excerpted from the study comapring hydroxyethyl starch (HES) versus Ringers acetate in severe sepsis. The Kaplan-‐Meier curve demonstrates that the effect between Riger‘s acetate and HES 130/0.42 is minimal (if any) during the first 20 days of treatment. However, by day 90 after the initiation of treatment there is a substantially higher survival rates in the group receiving Ringer‘s solutionversus the group that was administered HES 130/0.42. 15