© 2019 Parexel International Corporation
Don’t look back in anger –
learnings from a sample
size re-estimation
Emma Crawford
3rd June 2019
© 2019 Parexel International Corporation
What will be discussed
Agenda
About me, the study and the sample
size
Why size matters
Regulatory guidance
Methodology
Discussion of results
Unblinded vs. Blinded re-estimation
My learnings and reflections
© 2019 Parexel International Corporation
Senior Biostatistician, Parexel, October 2016-Present
Education
BSc Mathematics, University of Greenwich, 2009-2013
MSc Medical Statistics, LSHTM, 2013-2014
Working in industry since September 2014
Currently biostatistics lead for three studies
Active member of PSI CALC since July 2015
About me
© 2019 Parexel International Corporation
A Double-Blind, Randomized, Placebo Controlled Study of the Efficacy and Safety
of Three Doses of Drug X in Subjects with Condition A
Phase 2
Randomization ratio 1:1:1:1
Primary endpoint: Change from baseline to Week 12 in Measure 1
Measure 1 analysed on log scale
About the study
Drug X – Low dose
Drug X – Medium dose
Drug X – High dose
© 2019 Parexel International Corporation
Two-sided type I error probability = 0.05
Treatment difference = -0.141 (log scale)
SD of 0.261
80% power for each pairwise comparison
10% drop out rate
Approx. 244 subjects randomized (220 completers, 55 per arm)
About the sample size – original assumptions
© 2019 Parexel International Corporation
When designing clinical trials, size matters
More specifically the number of subjects randomized
Too few subjects
May not be able to answer the research question posed
Too many subjects
Waste resources
Expose more subjects than needed to an inferior or useless/futile treatment
Why size matters…
© 2019 Parexel International Corporation
ICH E9
Section 4.4 “Sample size adjustment”
February 1998
EMA – Reflection paper on methodological issues in confirmatory clinical trials planned with adaptive
design
Section 4.2 “Sample size reassessment”
October 2007
FDA – Adaptive Designs for Clinical Trials and Drugs and Biologics, Guidance for Industry
Section V.B. “Adaptations to the Sample Size”
September 2018
Regulatory documentation
© 2019 Parexel International Corporation8
Regulatory guidance
Blinded re-estimations
Uncertainty
Type 1 error
Prospective planning
No risk to maintaining trial
integrity
Blinded re-estimations
Unblinded re-estimations
Uncertainty
Type 1 error
Prospective planning
Unblinded re-estimations
Uncertainty
Power
Prespecified rules
Type 1 error
Prospective planning
Challenges in maintaining
trial integrity
ICH (1998) EMA (2007)
FDA (2018)
© 2019 Parexel International Corporation
Primary endpoint data reviewed to check assumption for SD
Done on a fully blinded basis
Use an overall pooled estimate of SD
May result in an increase in sample size, no decrease
Up to 320 subjects (approx. 80 per group) may be recruited
Precise number documented in a protocol amendment
About The sample size – re-estimation
© 2019 Parexel International Corporation
Mixed model for repeated measures (MMRM)
PROC MIXED in SAS 9.3
Obtained pooled SD estimate
PROC UNIVARIATE in SAS 9.3
SD of residuals
Re-ran original sample size calculation using re-estimated SD
Methodology
© 2019 Parexel International Corporation
Scenario α
Power
δ (log)
δ (abs)
σ
Total Sample Size
*
Original assumption
(Reductions: 10%
Placebo, 35% Active)
0.05
80%
-0.141
-25%
0.261
244
Re-estimated SD
0.05
80%
-0.141
-25%
0.371
484
Re-estimated power for 320 subjects
0.05
63%
-0.141
-25%
0.371
320
Re-estimated power for no change
0.05
51%
-0.141
-25%
0.371
244
Reductions: 20% Placebo, 45% Active
0.05
80%
-0.163
-25%
0.371
364
Reductions: 20% Placebo, 42% Active
0.05
80%
-0.141
-22%
0.371
484
Reductions: 35% Placebo, 60% Active
0.05
80%
-0.211
-25%
0.371
220
Reductions: 35% Placebo, 53% Active
0.05
80%
-0.141
-18%
0.371
484
Sample size re-estimate #1
* Allowing for 10% drop-out rate
© 2019 Parexel International Corporation
Increase in SD = increase from original sample size
If sample size is not increased as required:
Retain type 1 error = power decreases
Explored affect of assumption changes on sample size
Increase placebo response rate & same log difference = smaller absolute difference &
same sample size as re-estimate
Increase placebo response & same absolute difference = larger log difference &
decrease sample size from re-estimate
Further increase placebo response & same absolute difference = larger log difference &
decrease sample size from original
Discussion of results: re-estimation #1
© 2019 Parexel International Corporation
Logistical restrictions may hinder results of re-estimation
Realized the limitations of a blinded sample size re-estimation
Unable to confirm assumptions for placebo response and log treatment difference
Learnings from re-estimation #1
© 2019 Parexel International Corporation
Scenario α
Pow
er
δ (log)
δ
(abs)
σ Total
Sample
Size ^
Original assumption
(Reductions: 10%
Placebo,
35% Active)
0.05
80%
-0.141
-25%
0.261
244
Re-estimation #1
0.05
80%
-0.141
-25%
0.371
484
Re-estimated SD
0.05
80%
-0.141
-25%
0.314
348
Sample size re-estimate #2
^ Allowing for 10% drop-out rate
© 2019 Parexel International Corporation
Still an increase in SD from original assumption
SD is smaller compared to re-estimation #1
Discussion of results: re-estimation #2
© 2019 Parexel International Corporation
Timing is key
Still bound by limitations of a blinded sample size re-estimation
Still unable to confirm assumptions for placebo response and log treatment difference
Could consider a different method for sample size re-calculation
Learnings from re-estimation #2
© 2019 Parexel International Corporation
Blinded – Pros
Preserves study blinding
Negligible impact on type 1 error
Check assumption of pooled
variability
Unblinded vs. blinded sample size re-estimation
Blinded – Cons
Conservative approach
Unable to check assumptions on
treatment effect
Can lead to over-estimation of
SD
Could lead to a larger sample
size than is needed
© 2019 Parexel International Corporation
Unblinded – Pros
Check assumption of treatment
effect
Can combine with an early
termination rule
Unblinded vs. blinded sample size re-estimation
Unblinded – Cons
May require a weighted final
analysis to avoid inflation of type I
error probability and bias
Increases risk of accidental
unblinding
Decision could inform blinded
study team of interim results
Data Monitoring Committee required
© 2019 Parexel International Corporation
Consider a modern dose-finding study design (MCP-Mod)
Plan an adaptive interim analysis in the protocol
Allow for early stopping and sample size re-estimation
Perform unblinded re-estimate at time when sufficient data is available
Use a predictive/ conditional power criteria
Include a futility analysis to allow the study to stop early if required
Reflection – if we were to re-do the study….
© 2019 Parexel International Corporation
Ensure correct planning of any re-estimation
Are the study assumptions valid?
Consider the logistics not just the statistics
To summarise…
© 2019 Parexel International Corporation
Thank you
© 2019 Parexel International Corporation
Back-up
© 2019 Parexel International Corporation
Kieser M, Friede T. Blinded sample size re-estimation in multi-armed clinical trials. Drug
Information Journal 2000; 34:455– 460.
Kieser M, Friede T. Simple procedures for blinded sample size adjustment that do not
affect the type 1 error rate. Statist. Med 2003; 22:3571-3581
https://www.fda.gov/downloads/Drugs/.../Guidances/UCM201790.pdf
http://www.ich.org/fileadmin/Public_Web_Site/ICH_Products/Guidelines/Efficacy/E9/Ste
p4/E9_Guideline.pdf
https://www.ema.europa.eu/en/documents/scientific-guideline/reflection-paper-
methodological-issues-confirmatory-clinical-trials-planned-adaptive-design_en.pdf
References
© 2019 Parexel International Corporation
US*
UK
Total
Screening
68
21
89
Week 2
64
16
80
Week 4
56
11
67
Week 12
34
1
35
Number of subjects included in analysis
#1
© 2019 Parexel International Corporation
US*
UK
Total
Screening
112
65
177
Week 2
97
56
153
Week 4
93
49
143
Week 12
68
30
98
Number of subjects included in analysis
#2
