1. No category

University of Dundee
School of Medicine
Protocol Development and
Statistical Analysis Plans
Petra Rauchhaus
TCTU Clinical Trials Statistician
Importance of the Protocol
Funders
Trialists
Ethics
Journals
Patients
• Provide rationale for the trial
• Define trial goals and processes
• Define methods of analysis/ reporting
• Enable scientific and ethical review
• Provide a “Trial Roadmap”
Systematic
Reviewers
Healthcare
providers
Policy makers
Journals
Ethics
Importance of the Protocol
• GCP Requirement
• Ethics Committee requires a protocol for
submission
• Part of the EU Clinical Trials Register (EUDRACT)
• Ensures in Multi-Centre Trials that all centres
perform the study in the same way
• Journals require a registered protocol for
publication
• Not only for CTIMPS, Non-CTIMPS also benefit
from a good protocol
What could go wrong?
• Missing details of basic trial design (uncontrolled/
controlled/ randomized)
• Imprecise or missing description of the primary
outcome in the protocol
• Sample Size calculation not reported
• Limited methodological information
• Interventions not well defined
• Planned subgroup analyses missing
• Favourable reporting of positive outcomes
• Adverse events suppressed in reports
Lack of general information
70
60
% Inadequate information
59
50
40
40
41
34
30
25
20
10
0
Allocation
Concealment
Blinding
Primary
Outcome
Power
Calc.
Chan AW et al, BMJ 2008; Al-Marzouki S et al, Lancet 2008
Adverse
Events
Reporting
System
Lack of statistical information
Primary
Outcome
Analysis
Handling
Missing
Data
Handling
deviations
Adjusted
Analyses
Chan AW et al, BMJ 2008; Al-Marzouki S et al, Lancet 2008
Subgroup
Analyses
Protocol standards
There is a number of support documents:
•
ICH Guideline E6 defines the protocol structure (15 sections
with several sub-points each)
•
SPIRIT (Standard Protocol Items for Randomized Trials)
initiative by statisticians, journal editors and PIs
•
CONSORT guidelines to report trials
•
EQUATOR Network:
http://www.equator-network.org/
•
TASC SOP 14: Writing a protocol
•
Protocol Template on the TASC website:
http://www.tasc-research.org.uk/_page.php?id=208
Definition of a protocol
Pre-Trial Document containing transparent
description of:
• Background and objectives
• Population and interventions
• Methods and statistical analysis
• Ethical and administrative aspects
Title
• A title uniquely identifies the project
• It should summarize the aim and methods of the
trial
• Important information (e.g. randomized, doubleblind, parallel group) should be included in the
title
• Indexers on websites such as PubMed may not
classify a report correctly if the authors do not
explicitly report information in the title
• “A Prospective Randomized Study of Medial
Patellofemoral Ligament (MPFL) Reconstruction “
Synopsis
• Brief overview over the study aims and
conduct
• Should contain sufficient information about a
trial to serve as an accurate record of its
conduct
• Should accurately reflect what is included in
the full protocol and should not include
information that does not appear in the body
Background
• The Declaration of Helsinki states that biomedical
research involving people should be based on a
thorough knowledge of the scientific literature
• Thus, the need for a new trial should be justified in the
introduction
• Explain the scientific background and rationale for the
trial
• Report any evidence of the benefits and harms
• Ideally, it should include a reference to a systematic
review of previous similar trials or a note of the
absence of such trials
Objectives
• Objectives are statements what the researcher
means to do
• Objectives can be seen as smaller problems in the
larger research area
• E.g. “Improving cancer care” is a large research
area which is too broad to be tested within a trial.
Impact of physiotherapy on QOL of late stage lung
cancer patients is testable within a trial.
• Ensure that objectives are specific, measurable and
clinically important
• Changing objectives can sometimes make a trial
better
Outcomes
• Is the measurable part of the objective
• Ensure that the outcome is appropriate to the
objective it serves.
• Define clearly what the outcome is and how it will be
measured
• If outcomes are measured several times, specify
time point of interest
• If possible, use validated and measurable outcomes
• If there is more than one assessor, state how many
there are and how discrepancies in measurement
will be handled
Trial Design
• Define the type of trial, e.g. parallel group, crossover or factorial
• Define the conceptual framework, e.g. superiority,
non-inferiority, equivalence or other
• If a less common design is employed, authors are
encouraged to explain their choice
• This is especially important because it might have
implications on sample size or analysis
• Include allocation ratio if more than one group, and
unit of allocation (patient, practice, lesion)
Eligibility Criteria
• Should be well defined and appropriate to the trial
• Define which patient groups are involved and how
they relate to the objectives
• Eligibility criteria which are too narrow can
jeopardize the study
• Eligibility criteria too wide can invalidate the
outcomes
• E.g.: Including Stage IV Cancer patients in a study
examining the effectiveness of two different
treatments might fail, as the diseases is too
advanced already to make a difference
Sites and Locations
• Goes hand in hand with the eligibility criteria, as
certain subjects need certain locations
• E.g.: primary care, hospital wards, specialized units
• Healthcare institutions vary in their organisation,
experience, and resources
• Social, economic, and cultural environment and the
climate may also affect a study’s validity
• Especially important in multicentre trials,
particularly in international studies
Interventions
• Describe all interventions including controls in
great detail
• It must be possible to be reproduced if necessary
• If you compare to “usual practice” describe what
that means, do not assume everyone knows
• If interventions are variable, e.g. adaptation of
radiation doses or drug regimes, define rules of
application
• In dose-escalation studies, define stopping rules
Sample Size
• Sample size calculations are based on previous
trials measuring the outcome
• Ensure that the patient population matches the trial
population
• Where no previous trials are available, sample size
is often based on assumptions
• Sample size is only as accurate as the assumptions
• Where more than one outcome is present, sample
size is calculated for the primary outcome usually
• It is possible to use the largest sample size to get
the best power
Interim Analysis
• Interim Analysis can diminish the trial power
• Error rates increase as the number of analysis
increases
• E.g. doing 5 interim analysis requires a p-value of
0.01 rather than 0.05, and can give an error rate of
19% rather than 5%
• Use only when necessary
• Some trials require interim analysis, e.g. for a DMC
• If possible, separate the DMC analysis from the
main analysis
Randomisation
• Randomized trials are the gold standard
• Randomization requires a program to be written
• Sequence generation must be reproducible at any
stage
• Define criteria for stratification and minimisation
• Try to avoid predictable block sizes
• If possible, blinding should be employed
• Blinded studies require an independent statistician
• Minimization is dynamic and therefore less
predictable
Allocation
• Allocation concealment is not blinding
• Define how the allocation is applied to the subjects
• Define how to conceal allocation until the subject is
included into the trial
• Ensure that the person doing the screening is not
familiar with the allocation sequence
• Decide whether to include a subject into the trial
before the allocation
• If possible, use a third party to allocate subjects
Statistical Analysis
• Statistical analysis must be described
• Descriptive statistics should be defined for an
overview over the data
• Define the appropriate methods for the data
• Describe briefly missing or spurious data
• Keep the description of the statistical analysis short
• Mention checks of normality and independence
• Do not hesitate to involve a statistician with this
part of the protocol
• A detailed statistical analysis plan (SAP) should be
written during the course of the trial
Statistical Analysis Plan (SAP)
Statistical Analysis Plan (SAP)
• It is critical link between the conduct of
the clinical trial and the clinical study report.
• General statistical analysis is defined in the
clinical protocol
• The SAP contains a more technical and
detailed elaboration of the analysis
• Recommended by the CONSORT guidelines
and ICH Guideline E9 (Statistical Principles
for Clinical Trials)
Why write an SAP?
Establish Good Statistical
Practice
Implement the Trial as outlined
in the Protocol
Study Methods
(Data Collection
Trial conduct)
Study Design
(Clinical Protocol)
Study Analysis provides
checks on the original design
Study Analysis
(SAP)
Analysis of the planned
study design, adapted to
the study methods
GCP requirements
• The statistical authorship of the SAP should be clear
• Version and date should be clearly defined
• The SAP should be reviewed/ updated immediately
before the blinded code is broken or before analysis
begins in an unblinded trial
• The SAP should be signed off by the PI/ CI and the
Statistician (and other members of the study team
where applicable)
• Changes in the SAP after study end should be
justified and fully documented in the statistical
report
When to write an SAP
• The SAP is written during the trial, after the clinical
protocol is final
• It must be finalized and signed off before the end of
the trial to avoid bias
• If the study is blinded, it must be finalized before the
blind is broken
• The SAP should be reviewed and possibly updated
as a result of the blind review of the data
• In adaptive trials, it must be finalized before the first
interim analysis
• Regulatory factors, such as a special protocol
assessment at the FDA, may affect the timing
Changes in Study Methods
• Protocol Amendments during the trial
• Change in the planned treatment (new developments
in therapy or guidelines)
• Recruitment does not go as planned
• Early termination of the trial can change patient
numbers
• Adding or removing a group
• Addition or removal of a planned test or procedure
• Changes in the outcomes or how they are measured
SAP Contents
• A brief description of the purpose
• The study rationale as laid out in the protocol
• Definition of analysis populations (usually ITT)
• How subject data will be summarized (descriptive
statistics or counts/ percents)
• Which statistical tests will be used on which data
• The statistical methods to be used for the endpoints
• When and how to impute missing or partial data
• Mocks (or shells) of all unique TLF's
• Quality control of the analysis
Writing an SAP
• Refer to TASC SOP 05 (Statistical Analysis Plans for
Clinical Trials of Investigational Medicinal Products)
http://www.tasc-research.org.uk/_page.php?id=266
• Follow the section headings laid out in the SOP
• Contact the study statistician if present
• If no study statistician is present, TASC statisticians
can review the SAP
• Distribute the SAP to all members of the study team
that can contribute
• Finalize the SAP before the study is finished
Benefits
• Clear Protocol and SAP show that a study was done
according to GCP standards
• Avoid biased analysis by defining the study
populations before study end
• Defined handling of missing data, outliers and data
deviations make the analysis more transparent
• Clearly defined subgroup analysis ward off data
dredging
• The study report and resulting papers will be more
likely to be of high quality
Any Questions?