Noninferiority & equivalence

Concepts and reasoning

The scientific question comes before the test

Suppose an established treatment works, but a new treatment is easier to administer. The new treatment need not be more effective to be useful. The question may be whether any loss of efficacy is small enough to accept in exchange for that practical advantage. Noninferiority formalizes this question by specifying the largest acceptable loss before examining the trial results. It does not establish that the treatments are identical, nor does it directly prove the practical advantage.

Define the treatment effect as new minus standard, with larger outcomes representing better health. A difference of −2 therefore favors the standard treatment. If the margin is 3 units, the unacceptable region is a true difference of −3 or lower. A positive margin is usually entered as the amount of acceptable loss; the null boundary on this effect scale is negative. For an adverse outcome, where smaller is better, the direction must be reversed. Writing the direction in words prevents a mathematically correct answer to the wrong clinical question.

Absence of evidence is not evidence of similarity

A superiority test asks whether the data rule out no difference. A wide interval such as −8 to 6 includes zero and gives a nonsignificant superiority result, but it also includes clinically important harm. Noninferiority instead asks whether the data rule out unacceptable harm. Equivalence is stricter in a different direction: it requires ruling out effects beyond both the lower and upper acceptable limits.

The margin is a scientific decision, not a convenient value that makes the sample affordable. Historical evidence for the active comparator, the reliability of that evidence in the new setting, and the clinical importance of the retained effect all matter. If the standard treatment would no longer work under the trial conditions, showing similarity to it would be uninformative. This is the role of assay sensitivity and the constancy assumption. FDA’s noninferiority guidance explains these design issues.

Mathematical foundations

On a small screen, scroll wide equations and tables horizontally.

Hypotheses and interval boundaries

Let and let be the acceptable loss. For an outcome where higher is better:

Rejecting the null at one-sided level is equivalent, for the usual continuous test, to placing the one-sided lower confidence bound above . At , that bound is the lower endpoint of the ordinary two-sided 95% interval. With symmetric equivalence limits, both one-sided null hypotheses must be rejected. At per test, the corresponding two-sided 90% interval must lie completely inside . Do not mix those confidence levels when comparing conclusions.

Why smaller margins require more information

For equal-sized independent groups with a common known standard deviation , a normal approximation gives the number per group:

Here is target power, is the standard-normal quantile, and is the true effect assumed for planning. The denominator is the squared distance from that assumed effect to the null boundary. If the new treatment is anticipated to be slightly worse, that distance shrinks and required enrollment increases. The formula presumes that the planned effect lies above the boundary. It is not an equivalence sample-size formula: equivalence power must account for satisfying both boundaries. Walker and Nowacki explain the distinctions.

Worked example

Hypothetical data for learning; independent of your analysis inputs.

Planning a hypothetical continuous-outcome trial

Assume an outcome standard deviation of 10 units, an acceptable loss of 3 units, equal true treatment means, one-sided alpha 0.025, and 80% power. The relevant normal quantiles are approximately 1.960 and 0.842. Substitution gives participants per group. Rounding upward gives 175 analyzable participants per group. With 10% loss to follow-up, suggests recruiting 195 per group under this simple attrition assumption.

Now suppose the investigators believe the new treatment might be 1 unit worse. The distance to the boundary becomes instead of 3. The unrounded requirement increases by a factor of . Assuming exact equality was therefore an influential planning choice, not a neutral default.

After a hypothetical trial, consider these intervals for new minus standard, all calculated using the confidence level appropriate for their stated test:

The first conclusion is a statement about the ruled-out loss. It should be reported with the chosen margin and interval, not simply as “the treatments are the same.”

Interpreting the results

Read the interval against a prespecified boundary

For a beneficial outcome, define the effect as treatment minus control and let the positive margin be Δ. Noninferiority requires the lower confidence bound to exceed −Δ. A point estimate above −Δ is insufficient: uncertainty must also exclude an unacceptable loss. At one-sided alpha 0.025, the corresponding lower bound is the lower end of a two-sided 95% confidence interval. The confidence level follows the testing rule; it is not chosen after seeing the result.

For equivalence, both ends of the interval must lie strictly inside −Δ and +Δ. Two one-sided tests at alpha 0.05 correspond to a two-sided 90% interval. An interval that overlaps a margin is inconclusive for that objective, even if its center is zero. A nonsignificant superiority test also leaves equivalence unresolved.

Separate statistical conclusions from practical decisions

An interval of −2 to +1 with Δ = 3 rules out losses of 3 or more but permits a loss near 2. Report that remaining uncertainty alongside any advantages in safety, convenience, or cost. If the entire interval is above zero, superiority may also be supported, subject to the prespecified testing strategy. If the interval crosses the noninferiority boundary, failure to demonstrate noninferiority does not by itself establish inferiority.

A sample-size calculation answers a different question: how often the planned test would succeed under particular assumed effects and variability. Target power is not the probability that treatments are equivalent, and achieved planning power is not evidence from a completed trial.

Assumptions and limitations

Define the comparison before collecting data

Specify the endpoint, follow-up time, population, effect scale, and direction of benefit. A margin of 0.10 on a risk-difference scale is ten percentage points; it is not a 10% relative change. For adverse outcomes, smaller values are better, so the inequality and effect coding must be reconsidered explicitly. A margin in raw measurement units cannot be reused unchanged for a standardized difference or ratio.

Justify what loss is acceptable

The margin needs both clinical justification and evidence that the active comparator has an effect worth preserving. Historical studies should be relevant to the current population, endpoint, background care, and adherence. The constancy assumption concerns whether that historical effect remains applicable; assay sensitivity concerns whether the current trial could distinguish an effective from an ineffective treatment. These are substantive design questions, not properties established by a small standard error. FDA's noninferiority guidance explains these requirements and the distinction between the comparator effect and the smaller loss considered acceptable.

Match the design to the calculation

A simple two-group normal approximation assumes independent observations and a credible variance model. Small binary event counts, clustering, repeated measurements, unequal variances, or an effect near a probability boundary may need other methods. Nonadherence, switching, and missing outcomes can alter the treatment effect being estimated. Define how such events enter the estimand and analysis rather than assuming that either intention-to-treat or per-protocol analysis is automatically conservative.

Diagnostics and sensitivity analysis

Audit the statistical decision

Check the subtraction order, units, confidence level, margin, and inequality together. Draw the confidence interval and mark zero and each relevant margin. This exposes sign errors and distinguishes noninferiority, equivalence, superiority, and inconclusive results. Do not use rounded bounds to decide a borderline result.

Examine sensitivity before committing to recruitment

Repeat planning calculations over plausible treatment effects and standard deviations, or plausible response probabilities for a binary endpoint. For a means comparison, required sample size is approximately inversely proportional to the squared distance between the assumed effect and the null boundary. Moving an expected difference from 0 to −1 with margin 3 reduces that distance from 3 to 2 and increases the unrounded requirement by roughly 2.25. Dropout inflation addresses the expected number of analyzable participants; it does not remove bias from informative missingness.

Review the eventual trial, not just the formula

Inspect recruitment, adherence, missingness, and outcome ascertainment by group. Explain differences between analyses targeting different estimands. Assess whether reasonable missing-data or protocol-deviation assumptions change the conclusion, with the margin fixed as prespecified. A narrow interval from a poorly conducted trial cannot restore assay sensitivity. Report the margin rationale, analysis population, effect estimate, interval, and sensitivity results so readers can assess what loss was actually ruled out.

References and other material

Read the methodological references and supporting tutorials below. The examples in this lesson are original teaching calculations, not findings from these sources.

• FDA: non-inferiority clinical trials — Margin selection and interpretation for drug and biologic trials.

Related method guides

• Cluster randomized trial
• Survival trial sample size
• Browse all analysis families