Propensity weighting
Estimate a treatment mean difference for a chosen target population using propensity-based weights.
Concepts and reasoning
Constructing a comparable population
Weighting changes how much each observation contributes. Consider a treated person whose baseline characteristics usually lead to control. That person supplies information about a relatively uncommon treated experience. Inverse probability weighting increases their contribution so the observed treatment group can represent a broader target population.
This is often described as constructing a pseudo-population in which measured baseline covariates no longer predict treatment. It does not create new patients or new information. Large weights make an estimate depend heavily on a few people and can increase uncertainty.
The target matters. ATE weights represent the full eligible population; ATT weights represent the treated population. Overlap weights emphasize people whose baseline characteristics make either treatment plausible. These are different scientific questions, so a more stable estimate obtained by changing weights may also answer a different question.
Design first, estimate second
Estimate treatment probabilities from pretreatment covariates, construct weights for the chosen target, and inspect balance before interpreting outcomes. A well-fitting treatment prediction model is not enough. Evaluate weighted covariate means and distributions, the distribution of weights, and the number of observations carrying most of the influence.
The WeightIt tutorial illustrates target-specific weights and balance assessment. Weighting is attractive because it can use many observations while making the target explicit, but it needs strong overlap and a defensible treatment-assignment model.
Mathematical foundations
Three target populations
Let A be 1 for treatment and 0 for control, and let e(X) = P(A = 1 | X). Common unstabilized weights are:
| Target | Treated weight | Control weight |
|---|---|---|
| ATE | 1/e(X) | 1/[1 − e(X)] |
| ATT | 1 | e(X)/[1 − e(X)] |
| Overlap population | 1 − e(X) | e(X) |
For an ATE contrast, a normalized, or Hájek, estimator compares weighted group means:
Normalization avoids treating a random sum of estimated weights as an exact population size. It differs in finite samples from dividing an inverse-probability sum by the original sample size.
A useful concentration diagnostic within each treatment arm is
Equal weights give ESS equal to the number of observations. Unequal weights reduce it. ESS is not the degrees of freedom for every weighted analysis and does not measure residual bias. Multiplying every weight in an arm by the same positive constant leaves both its normalized mean and ESS unchanged.
Worked example
Worked example: an influential observation
Three treated people have propensity scores 0.5, 0.5, and 0.1, and outcomes 10, 12, and 20. Their ATE weights are 2, 2, and 10. The ordinary mean is 14, but the weighted mean is
The third person supplies 10/14, or 71.4%, of the total weight. The arm's ESS is 14²/(4 + 4 + 100) = 1.815, despite having three observations. This does not mean the person is erroneous. It means the target comparison is poorly represented by the available treated sample.
Suppose a suitable weighted control mean is 13. The estimated contrast is 4.429 outcome units. Removing or capping the third person's weight would substantially change it. Such sensitivity should prompt an overlap review, not an automatic choice of whichever cap produces the preferred result.
For a control person with e(X) = 0.8, ATE, ATT, and overlap weights are respectively 5, 4, and 0.8. For a treated person with the same score they are 1.25, 1, and 0.2. The overlap scheme downweights people who almost always receive their observed treatment because they are less informative about a choice between treatments.
Interpreting the results
The weights define the target population
Average treatment effect weights target the supplied population; average treatment effect on the treated weights target treated participants; overlap weights emphasize people with a meaningful chance of either treatment. These answer different questions and can yield different effects when treatment effects vary.
The weighted mean difference is treatment minus control on the outcome scale. For a binary endpoint, it is an absolute risk difference. A treatment-weighted population is a statistical construction, not a new set of observed participants.
Effective sample size summarizes the concentration of weights. A large raw sample with a few dominant observations can have a much smaller effective size and unstable estimation. A bootstrap interval describes sampling uncertainty under the estimation procedure and data assumptions. It does not incorporate every possible bias from unmeasured confounding or an inappropriate target population.
Assumptions and limitations
Adequate overlap is essential
Weighting requires conditional exchangeability given an appropriate set of pretreatment confounders, consistency of the intervention, and positivity for the target comparison. When some covariate profiles almost always receive one treatment, the other outcome is weakly supported and inverse-probability weights can become extreme.
The propensity model must adequately represent treatment assignment for the intended weighting estimator. Good prediction of treatment is not the objective by itself: nearly perfect prediction can reveal a lack of comparable observations. Balance in outcome-relevant covariates matters more than a classification accuracy score.
Independent records and appropriate outcome/time definitions are assumed by a simple person-level bootstrap. Repeated measures, clusters, time-varying treatments, and informative censoring require additional structure. Trimming or changing the weighting target changes the estimand and should be justified explicitly.
Diagnostics and sensitivity analysis
Inspect balance and weight concentration together
Review weighted standardized mean differences, propensity overlap, largest weights, and effective sample sizes by arm. Excellent mean balance with a tiny effective sample can still be fragile. Conversely, stable weights do not ensure adequate adjustment if important covariates remain imbalanced.
Investigate whether extreme weights reflect coding errors, an overly restrictive population, or genuine treatment separation. Prespecified model refinements or a different target population may be reasonable, but should not be chosen according to the outcome estimate. The WeightIt tutorial illustrates evaluating balance and effective size jointly.
Check bootstrap success and interval stability across sufficient replications. Repeated fitting failures signal weak information or model instability; simply reporting only the successful estimates can hide the problem. Add sensitivity analyses for unmeasured confounding and defensible adjustment choices.
References and other material
Read the methodological references and supporting tutorials below. The examples in this lesson are original teaching calculations, not findings from these sources.
- WeightIt: estimating balancing weights — Worked examples of weighting, balance and effective sample size.