Difference-in-differences
Compare outcome changes before and after an intervention between treated and comparison groups.
Concepts and reasoning
Subtracting a background change
A clinic introduces a program and its average outcome improves. The before–after difference is not necessarily caused by the program: seasonality, economic changes, or another policy may have affected everyone. Difference-in-differences, DID, uses the change in a comparison group to estimate that background movement.
The basic design contains a treated group and a comparison group, observed before and after treatment begins. First calculate the change in each group. Then subtract the comparison change from the treated change. Groups need not start at the same outcome level; the key assumption concerns how their outcomes would have changed without treatment.
The missing untreated trend
Parallel trends means that, absent the intervention, the average change for the treated group would have matched the comparison group's change. The untreated post-intervention mean in the treated group is unobserved. Parallel trends supplies that missing counterfactual.
This is more precise than saying that “the groups are similar.” Two groups can have different baseline means but similar untreated trends. Conversely, groups with identical baseline means can have very different trends. Covariate adjustment may make a conditional parallel-trends assumption more plausible, but it does not automatically establish it.
The difference-in-differences chapter in The Effect emphasizes that design credibility comes from the comparison and timing, not just the interaction coefficient. A convincing analysis explains why the comparison group captures the shocks the treated group would otherwise have experienced.
Mathematical foundations
The two-by-two contrast
Let T and C denote treated and comparison groups, and 0 and 1 denote pre and post periods:
In a simple linear regression, the same contrast is the interaction coefficient:
Here β₁ captures the baseline group difference, β₂ the comparison-group time change, and τ the additional treated-group change. With repeated observations on the same people, residuals are correlated within person; the regression does not make them independent.
Parallel trends can be written using untreated potential outcomes:
Together with no anticipation, consistency, and an appropriate comparison design, this identifies the average treatment effect for the treated group in the post period. It is an additive-scale assumption. Parallel changes in outcome units are different from parallel percentage changes or parallel log outcomes.
Worked example
Worked example: constructing the counterfactual
Suppose the hypothetical mean outcome is:
| Group | Before | After | Change |
|---|---|---|---|
| Treated | 10 | 15 | 5 |
| Comparison | 8 | 10 | 2 |
The before–after treated change is 5. The DID estimate is 5 − 2 = 3 outcome units. Under parallel trends, the treated group's untreated post mean would have been 10 + 2 = 12. Its observed mean is 15, leaving an estimated intervention contribution of 3.
The regression coefficients are β₀ = 8, β₁ = 2, β₂ = 2, and τ = 3. Substituting T = 1 and Post = 1 gives 8 + 2 + 2 + 3 = 15. This reconstruction is a useful way to understand the interaction term.
Now suppose a separate funding increase affected only treated clinics at the same time. The extra change could be caused by the program, the funding, or both. DID cannot distinguish them using these four means. Likewise, if the program changed who remained in the observed sample, the apparent effect might reflect changing composition.
Finally, suppose there are 1,000 patients but only two clinics, one treated and one comparison. The intervention has two independent assignment units, not 1,000. A small patient-level standard error would overstate what the design can establish about a clinic-level intervention.
Interpreting the results
Interpret a difference in changes
Difference-in-differences subtracts the comparison group's before-to-after change from the treated group's change. If treated outcomes rise by 8 and comparison outcomes rise by 3, the estimate is 5 outcome units. The comparison group's change supplies the estimated counterfactual trend, not its outcome level.
The effect is the additional change associated with treatment under the identification assumptions. Baseline group means may differ without invalidating the design; the crucial question concerns how they would have changed without treatment. Conversely, equal baseline means do not establish parallel untreated trends.
The interval should reflect dependence within the units followed over time and the level at which common shocks or treatment assignment occur. A two-period difference does not describe dynamic effects, anticipation, or the trajectory of treatment response. State the treatment timing and the follow-up interval with the estimate.
Assumptions and limitations
Parallel trends concerns an unobserved counterfactual
Without treatment, the treated group's mean change must have matched the comparison group's mean change on the chosen outcome scale. This cannot be verified from the post-treatment observations because the untreated outcome for treated people is missing. Historical trends can inform the assumption but cannot prove it.
No anticipation means behavior or outcomes were not already affected before the recorded intervention. Spillovers into the comparison group undermine its role as an untreated counterfactual. Changes in sample composition, concurrent policies, or differential measurement can also imitate a treatment effect.
The simple balanced-panel design follows the same individuals in one pre and one post period with fixed group membership. Multiple treatment dates, repeated cross-sections, many periods, and effects that vary across cohorts require extensions rather than mechanically applying the same interaction coefficient.
Diagnostics and sensitivity analysis
Investigate alternative explanations for the change
Plot all four group-period means and verify the arithmetic of the two changes. Review the intervention date, outcome definition, and concurrent events that may affect one group differently. A visibly different post-treatment slope is the quantity to explain, not evidence by itself that parallel trends held.
With only one pre period, there is no pretrend to test. If additional historical data exist, examine them in an appropriate design and assess plausible anticipation or differential trends. Use negative-control outcomes or alternative comparison groups only when their substantive role is justified.
Check whether uncertainty is clustered at the treatment-assignment or shared-shock level. Many people within a few policy units do not create many independent policy comparisons. The Effect's difference-in-differences chapter develops the counterfactual interpretation and design limitations.
References and other material
Read the methodological references and supporting tutorials below. The examples in this lesson are original teaching calculations, not findings from these sources.
- The Effect: difference-in-differences — An author-published, free chapter on the identifying assumptions and examples.