Propensity score matching
Compare outcomes among treated participants and similar untreated participants in observational data.
Concepts and reasoning
The missing comparison
Suppose people receiving a new treatment are younger and have milder disease than those receiving usual care. A difference in outcomes combines the treatment association with differences in prognosis. For each person we would like to compare the outcome under treatment with the outcome under control, but observe only one. Matching attempts to construct a credible comparison using other people with similar pretreatment characteristics.
Write the potential outcomes as Y(1) and Y(0). The average treatment effect among the treated, ATT, is E[Y(1) − Y(0) | A = 1], where A indicates treatment. It asks what would have happened to the treated population had it instead received control. This differs from the average treatment effect in the entire eligible population, ATE. Decide which question matters before discarding observations.
A balancing score, not a prognosis score
The propensity score e(X) is the probability of treatment given measured baseline covariates X. Among people with the same true propensity score, those covariates have the same distribution across treatment groups in the population. Matching on an estimated score can therefore reduce a high-dimensional comparison to a simpler one. It does not guarantee exact balance in a finite sample.
A logistic treatment model is a common way to estimate the score. Its purpose here is to support a balanced comparison, not to maximize classification accuracy. A treatment model that perfectly predicts treatment can reveal that no credible comparison exists. Include scientifically justified confounders; do not automatically include post-treatment variables, which can block part of the effect or introduce bias.
Nearest-neighbor matching chooses a control near each treated person. A caliper limits how far away a match may be. Matching without replacement prevents reuse of controls; matching with replacement may improve closeness but changes weights and uncertainty. Exact matching can additionally require agreement on critical categories. These choices trade closeness against retained sample size.
The MatchIt estimation vignette explains why the target population can change when treated observations are discarded. Matching is a design decision with consequences for the causal question, not merely a preprocessing convenience.
Mathematical foundations
Estimating an ATT from matched pairs
For M treated people with one matched control each, a simple matched difference is
This formula applies to equally weighted pairs. Variable-ratio matching or reused controls require the corresponding matching weights. Uncertainty must respect the design; treating every reused control as a new independent person exaggerates information.
A common continuous-covariate balance measure is the standardized mean difference:
Use a clearly specified standardizing denominator when comparing before and after matching; often the original sample's denominator is retained. An absolute SMD below 0.1 is a common diagnostic convention, not proof of no confounding. Examine distributions, variance ratios, important interactions, and scientific relevance as well. Balance tests based on p values can look reassuring simply because matching reduced sample size.
Worked example
Worked example: a matched outcome contrast
Consider an illustrative outcome where a higher value is better. Four treated people have plausible controls with similar baseline covariates:
| Pair | Treated outcome | Control outcome | Difference |
|---|---|---|---|
| 1 | 12 | 10 | 2 |
| 2 | 15 | 11 | 4 |
| 3 | 9 | 10 | −1 |
| 4 | 14 | 11 | 3 |
The estimate is (2 + 4 − 1 + 3) / 4 = 2 outcome units. This is a comparison of matched outcomes, not a comparison of propensity scores. The score is used to design the comparison.
Before matching, suppose mean age was 50 in treated people and 60 in controls, with a standardizing SD of 10. The SMD is −1.00. After matching, means are 50 and 51; using the same denominator gives −0.10. This is a substantial improvement, but other baseline variables still need assessment.
Now suppose there were originally five treated people, but a very frail treated person had no adequate control. The four-pair estimate describes the retained treated population. It cannot automatically answer the ATT for all five. Report the exclusion and examine how the retained population differs. A narrower, credible target is often more defensible than an apparently broad estimate based on implausible extrapolation.
Interpreting the results
Identify who the estimate represents
Matching constructs a comparison sample whose measured baseline characteristics are more similar across treatment groups. A mean within-pair outcome difference describes the retained matched sample. When treated people without suitable controls are excluded, the target becomes the matched treated subset rather than every treated person in the original population.
A positive outcome difference means treatment minus control is positive on the recorded outcome scale; whether that is desirable depends on the endpoint. For a binary outcome, the difference is an absolute probability difference, not an odds ratio.
A smaller standardized mean difference after matching is evidence of improved balance in that measured covariate. It is not evidence that unmeasured confounding is absent. Report the number retained, who was excluded, and the balance achieved alongside the effect, because a precise-looking summary without a target population is difficult to interpret.
Assumptions and limitations
Matching relies on design assumptions
Causal interpretation requires that measured baseline variables adequately address confounding, that each treatment is a meaningful and consistently defined intervention, and that comparable people could receive either treatment. Treatment, eligibility, and follow-up must be aligned in time to avoid biases such as counting guaranteed survival before treatment assignment.
Only appropriate pretreatment variables belong in the adjustment set. Including a variable affected by treatment can change the question or introduce bias. Selecting covariates by their treatment-prediction p-values alone does not establish an adequate confounding strategy.
A fitted propensity score is a summary of included covariates under a model. Similar scores need not imply close agreement on every covariate, and limited overlap cannot be fixed just by finding the nearest available control. Matching with replacement, variable ratios, or different distances can target different samples and require different analyses.
Diagnostics and sensitivity analysis
Assess the design before reading the outcome effect
Compare covariate balance before and after matching, focusing on meaningful differences rather than significance tests that change with sample size. Inspect distributions and important nonlinear terms where possible; matching means alone can leave differences in tails or interactions. MatchIt's balance guide explains why balance of the propensity score alone is insufficient.
Examine unmatched treated participants and whether they differ systematically from retained participants. Tightening a caliper can improve comparability while narrowing the population represented. Report both consequences rather than choosing the caliper that produces the preferred effect.
Check sensitivity to a prespecified set of reasonable matching choices and to plausible unmeasured confounding. If overlap remains poor, reconsider the population or comparison. A method cannot supply empirical support for a counterfactual treatment where no comparable observations exist.
References and other material
Read the methodological references and supporting tutorials below. The examples in this lesson are original teaching calculations, not findings from these sources.
- MatchIt: matching for causal inference — Matching choices, target populations and links to worked vignettes.