How to cra) the “Approach” sec1on of an R grant applica1on David Elashoﬀ, PhD Professor of Medicine and Biosta7s7cs Director, Department of Medicine Sta7s7cs Core Leader, CTSI Biosta7s7cs Program Overview • 
• 
• 
• 
• 
• 
Preliminary Data Study Design Sample Size and Power Analysis Sta7s7cal Methods Collaborators Wri7ng Strategies Preliminary Data •  Primary Ques7on: “Is there reason to believe that the study hypotheses could be true and is this research team capable of carrying out the study?” Necessary Elements: Preliminary Data •  Strong and relevant preliminary data key for R01 grants •  Demonstrate: –  Exper7se with assays –  Novel assays work in pa7ents/samples to be collected –  Support for hypotheses •  Use ﬁgures and tables where possible Ways to Fail: Preliminary Data • 
• 
• 
• 
• 
• 
Insuﬃcient annota7on for ﬁgures/tables Poor data analy7c techniques Weak support for hypotheses Unrealis7cally strong/naïve preliminary results Presen7ng needle in a haystack results Presen7ng too much preliminary data at expense of rest of the approach Study Design •  Primary Ques7on: “Is the design of the study appropriate to address the study aims?” Necessary Elements: Study Design •  What is overall study design (RCT, Cohort study, Case-‐Control, Cross-‐sec7onal, Biomarkers) •  Describe endpoints and clarify, if necessary, how they will be quan7ﬁed and their measurement scale. •  Describe study popula7on and control groups •  Inclusion/Exclusion Criteria •  Describe all study measures with appropriate measurement process details Addi7onal Considera7ons: Study Design •  Describe exis7ng popula7on clearly. -‐ Include relevant demographics -‐ Include informa7on on prognos7c or confounding measures. •  Nothing says that this is a ready to go study be^er than a clearly deﬁned popula7on that is relevant to the study aims. Addi7onal Considera7ons • 
• 
• 
• 
• 
• 
Randomiza7on methods for clinical trials Collect confounding factors How long will follow-‐up period be? Validity and reliability of study measures Subject matching? Valida7on of model building either with cross-‐
valida7on or training-‐test designs Ways to Fail: Study Design •  Study popula7on or design doesn’t match objec7ves •  Insuﬃcient 7me for recruitment and follow-‐
up. •  Lack of clarity with respect to availability of subjects •  Very uninteres7ng to read technical details of assays that are standard Sample Size •  Primary Ques7on: “Is the sample size suﬃcient to give the study the ability to answer the primary study ques7ons?” Necessary Elements: Sample Size •  Iden7fy study endpoint(s) for all aims. •  Clearly describe sample size for each aim •  For each endpoint: –  What is the eﬀect of interven7on or magnitude of the rela7onship? –  How much variability? –  Level of power? –  One or two sided test? –  What is the sta7s7cal test used to compute power? Addi7onal Considera7ons: Sample Size •  Account for study dropouts •  Account for mul7ple comparisons (either Bonferroni or False Discovery Rate) •  Ocen useful to examine sample sizes for a variety of scenarios when uncertainty exists concerning what is to be expected for an endpoint Ways to Fail: Sample Size •  No power analysis •  Sample size calcula7on does not have suﬃcient informa7on for a reviewer to replicate •  Sample size calcula7on does not use relevant preliminary data or methods described in the sta7s7cal analysis sec7on. •  Predic7on modeling with large number of predictors rela7ve to sample size •  Unrealis7c assump7ons about magnitude of eﬀect Bad Examples “A previous study in this area recruited 150 subjects and found highly signiﬁcant results (p=0.014), and therefore a similar sample size should be suﬃcient here.” “Our lab usually uses 10 mice per group.” “Sample sizes are not provided because there is no prior informa7on on which to base them.” "The throughput of the clinic is around 50 pa7ents a year, of whom 10% may refuse to take part in the study. Therefore over the 2 years of the study, the sample size will be 90 pa7ents. “ “It is es7mated that for a sample size consis7ng of 6 animals in each trial and with a tumor volume variance from 0.1 to 1.0 cm3 – that when the diﬀerence in the popula7on reaches 0.25, the power will reach 100%.” Good Examples “A sample size of 38 in each group will be suﬃcient to detect a diﬀerence of 5 points on the Beck scale of suicidal idea7on, assuming a standard devia7on of 7.7 points, a power of 80%, assuming a two sided signiﬁcance level of 5% and a two sample t-‐test. This number has been increased to 60 per group (total of 120), to allow for a predicted drop-‐out from treatment of around one third. This diﬀerence of 5 points is based on our prior study in which….. ” “A sample size of 292 babies (146 in each of the treatment and placebo groups) will be suﬃcient to detect a diﬀerence of 16% between groups in the sepsis rate at 14 days, with 80% power. This 16% diﬀerence represents the diﬀerence between a 50% sepsis rate in the placebo group and a 34% rate in the treatment group. This assumes a Chi-‐
square test with a two sided 0.05 signiﬁcance level. This es7mated diﬀerence in sepsis rate is based on the study of Bob et al [ref] in which they observed….” Sta7s7cal Methods •  Primary Ques7on: “Are the sta7s7cal methods appropriate for the analysis of the data that will be collected?” Necessary Elements: Sta7s7cal Methods •  Need methods sec7on for each aim. •  Clearly describe analy7c strategies for each endpoint. •  Methods should be appropriate for type of variable (ex. categorical, ordinal, count) and study design •  Typically includes inferen7al tes7ng of endpoints and model building Addi7onal Considera7ons: Sta7s7cal Methods •  Sta7s7cal methods appropriate for sample size (ex. Fisher test vs Chi-‐square test) •  Include evalua7on and valida7on strategies for regression/predic7on models •  Can include model assump7on checking methods •  Accoun7ng for missing data Ways to Fail: Sta7s7cal Methods •  Ignoring key confounders or demographic variables. •  Ignoring standard prognos7c or predic7ve measures in models •  Describing socware but not ideas/methods •  Analy7cal approach not appropriate for design and research ques7on Ways to Fail: Sta7s7cal Methods •  Ignoring key confounders or demographic variables. •  Ignoring standard prognos7c or predic7ve measures in models •  Describing socware but not ideas/methods •  Analy7cal approach not appropriate for design and research ques7on Collaborators •  Primary Ques7on: “Does the study have appropriate collaborators with suﬃcient eﬀort to perform the research described?” Necessary Elements: Collaborators •  Need an iden7ﬁed sta7s7cal collaborator with appropriate experience •  Biosketchs for faculty collaborator •  Budget jus7ﬁca7on for collaborator •  Le^er of Support if no funding is in applica7on. –  Make use of collaborators from on-‐campus service groups. (Ex: CTSI, Cancer Center) Addi7onal Considera7ons: Collaborators •  Staﬀ collaborator only need biosketch if no faculty on applica7on. •  Can include small % eﬀort for expensive faculty and larger % for staﬀ support. •  Make sure areas of weakness are covered with experienced collaborator •  Don’t include many collaborators with minimal eﬀort •  Not enough to men7on collaborators and write that they will take care of details Wri7ng Strategies •  Use the resources and human subjects sec7ons to full eﬀect –  Can give details of available study popula7on and subject demographics •  Standard experimental methods can be referenced •  Long blocks of text are boring a can ocen get skimmed. •  Emphasize key points: bold, underline Wri7ng Strategies •  Graphical displays: –  Theore7cal Framework –  Experimental Design –  Aims ﬂowchart –  Pa7ent characteris7cs –  Study measures Don’t Waste Space Grant Applica7ons Assistance •  Assistance with preparing grant applica7ons (CTSI) –  Study Design –  Data Analysis Protocols –  Sample Size and Power Analysis –  Budge7ng and Iden7fying Appropriate Collaborators –  Core facili7es •  Substan7al lead 7me with opportunity for mul7ple itera7ons is necessary for high quality grant applica7on assistance: Study Design vs Analysis sec7ons Final Thoughts •  Consult sta7s7cal collaborator for study design and approximate sample size some weeks in advance •  Most successful proposals require mul7ple itera7ons of research design sec7ons