Group-sequential & futility design
Explore a trial with planned interim analyses and possible early stopping for efficacy or futility.
Concepts and reasoning
A trial with planned opportunities to stop
A group-sequential trial evaluates accumulating evidence at prespecified interim analyses. It may stop early because evidence of benefit is strong or because achieving the objective has become unlikely. The design specifies when evidence will be reviewed and what constitutes a stopping signal.
Ordinary fixed-sample testing assumes one planned opportunity to reject. Repeatedly applying the same cutoff whenever results become available increases the chance of a false-positive conclusion. Interim statistics are correlated because later analyses include earlier participants, but that correlation does not remove the repeated-testing problem.
A group-sequential design calibrates a set of boundaries so that the probability of crossing any efficacy boundary under the null is controlled. An O'Brien–Fleming-style shape demands especially strong early evidence and reserves much of the error allowance for later analyses. A Pocock-style shape is more nearly constant across looks and generally spends more error early. The final numerical boundaries depend on the complete design.
Futility and information
A futility boundary allows stopping when evidence is insufficiently promising. A nonbinding futility rule is designed so that ignoring that recommendation does not invalidate efficacy error control under the specified design. Following it can still reduce power. A binding rule has a different calibration and should not be treated as optional.
Analyses are often scheduled by statistical information rather than calendar time. Information is approximately the inverse variance of an effect estimate. For survival outcomes it is commonly related to observed events; enrollment alone may not capture it.
The gsDesign introduction explains the canonical model. The FDA adaptive-design guidance discusses prespecification, trial integrity, and evaluating a design's operating characteristics.
Mathematical foundations
Correlated looks and overall error
At analysis j, let information be Iⱼ and the standardized statistic be Zⱼ. Under a canonical independent-increments model,
Here δ is the true effect on the modeled scale. The information fraction is tⱼ = Iⱼ/Imax. Efficacy boundaries bⱼ are chosen to satisfy
The event in this probability is a union across analyses. It is not the probability of crossing the last boundary alone.
If K simulations under the null produce R rejections, the estimated null rejection rate is R/K, with approximate Monte Carlo SE
This uncertainty measures simulation precision, not uncertainty about whether the trial's assumptions describe reality. More simulations improve precision but do not repair an inappropriate outcome model.
Worked example
Worked example: interpreting a simulated design
Suppose a hypothetical design permits stopping at total sample sizes 100, 200, and 300. At information fractions 1/3, 2/3, and 1, the canonical correlations are √(1/2) = 0.707 between the first two looks, √(1/3) = 0.577 between first and final, and √(2/3) = 0.816 between second and final.
Under one assumed alternative, simulations show the following probabilities of stopping for any reason:
| Stopping sample size | Probability | Contribution to expected N |
|---|---|---|
| 100 | 0.20 | 20 |
| 200 | 0.30 | 60 |
| 300 | 0.50 | 150 |
Expected N is 230, although the maximum is 300. This average cannot be interpreted without the corresponding efficacy and futility probabilities: a design that stops nearly everyone for futility can have low expected N and poor power.
Now run 10,000 independent null trials and observe 260 efficacy rejections. The estimated false-positive rate is 0.026. MCSE is approximately 0.00159, giving a rough 95% simulation interval of 0.0229 to 0.0291. A target alpha of 0.025 lies inside that interval. This is compatible with calibration but is not conclusive proof of exact error control.
For intuition about repeated testing, two independent tests each at 0.025 would reject at least once with probability 1 − 0.975² = 0.049375. Actual sequential looks are correlated, so that is not their error rate; it simply shows why a per-look alpha cannot be substituted for the design-wide alpha.
Interpreting the results
Interpret a trial policy rather than a single test
A group-sequential design defines when to inspect accumulating information and when to stop. Its overall false-positive probability concerns crossing an efficacy boundary at any planned look, not the nominal p-value at one look. Repeatedly applying an ordinary fixed-sample threshold would answer a different error-control problem.
Early efficacy boundaries can be stringent while later boundaries approach a conventional final threshold. A roughly constant boundary distributes the stopping opportunity differently. The choice affects the chance of stopping early and the expected sample size under each true effect.
A futility rule stops for insufficient promise. Failure to cross efficacy is not proof of no effect, and a futility stop is not a demonstrated equivalence result. Expected N averages over stopping paths; it is neither the maximum enrollment nor the sample size guaranteed for an actual trial.
Assumptions and limitations
Information timing and dependence matter
Interim test statistics are correlated because they reuse earlier observations. Boundary calibration must account for this dependence and the planned information fractions. Calendar dates and participant fractions approximate information fractions only under suitable outcome and accrual assumptions.
A simple canonical normal model assumes independent increments, a specified variance relationship, and a stable effect scale. Delayed outcomes, unequal allocation, changing variance, or population drift may alter the information process. A design calibrated for one endpoint should not be reused unchanged for another.
Nonbinding futility means efficacy error control is established without relying on compliance with the futility stop. Following futility can still reduce power. Operational procedures should preserve trial integrity and limit access to interim comparative results. The monitoring rules must be prespecified rather than adjusted in response to encouraging data.
Diagnostics and sensitivity analysis
Verify error control and operating characteristics separately
Inspect the planned information fractions and efficacy boundaries. Check numerical calibration under the null without futility, then simulate the complete design with its actual stopping rules. Differences between the calibrated alpha and simulated null success can be explained by futility and Monte Carlo error.
Evaluate several plausible effects, including smaller-than-expected benefits. Compare power, expected N, and the distribution of stopping times. A design that looks efficient at one large effect may have poor performance near the clinically relevant boundary.
Increase replications until Monte Carlo uncertainty is small enough for the design comparison. Do not tune rules to a favorable single seed. Document maximum enrollment, look timing, efficacy and futility rules, and uncertainty in estimated operating characteristics. Prespecified monitoring and reporting principles are discussed in FDA's adaptive-design guidance.
References and other material
Read the methodological references and supporting tutorials below. The examples in this lesson are original teaching calculations, not findings from these sources.
- FDA: adaptive designs for clinical trials of drugs and biologics — Planning adaptations and evaluating operating characteristics through simulation.