Difference-in-differences study
Did outcomes change more in an intervention group than in a comparison group?
When to use this project
Use difference-in-differences when outcomes are observed before and after an intervention in exposed and comparison groups, and a credible argument can be made that their untreated trends would have changed similarly without the intervention (parallel untreated trends).
How it differs
Unlike a simple before–after study, it subtracts change in a comparison group. Unlike a basic interrupted time series, its identifying argument is about comparative untreated trends. Staggered adoption requires methods suited to treatment timing and heterogeneous effects.
Decisions before you start
- Define adoption units, timing, eligible controls and the target effect.
- Distinguish longitudinal panels from repeated cross-sectional samples.
- Assess no anticipation, interference, composition changes and parallel trends; a nonsignificant pre-trend test does not establish the identifying assumption.
How the flows work together
Each flow below links to its diagram, steps and tables. Flow order organizes the project; it does not imply that all work is strictly sequential. Data assessment, analysis planning and protocol refinement may inform one another.
1. Comparative change overview
Overview · Question through comparative change interpretation
Use this overview to state the purpose and link to detailed decisions. Keep a versioned protocol and decision register, and add evidence references for the rationale.
2. Groups and adoption timing
Selection · Define units treatment and periods
Define who adopts the intervention, when adoption occurs and which units are eligible comparisons. Investigate anticipation, spillovers and differential external shocks before estimation.
3. Panel data preparation
Data · Align repeated measurements and covariates
Panels follow the same units; repeated cross-sections may contain different people. Both require comparable outcomes and attention to composition changes. Do not control automatically for variables affected by treatment.
4. Identification and estimation plan
Statistics · Specify comparison assumptions and estimands
With staggered adoption and heterogeneous effects, a conventional two-way fixed-effects coefficient can combine unsuitable comparisons. Define group-time effects, eligible controls and aggregation, with assignment-level inference.
5. Diagnostics and sensitivity
Statistics · Assess trends anticipation and spillovers
Pre-intervention patterns help assess plausibility but cannot verify unobserved post-intervention parallel trends. Investigate anticipation, composition, alternative controls and sensitivity to plausible departures.
Equations and statistical interpretation
These explanations describe notation and assumptions. Flow does not calculate these values.
Two-group, two-period contrast
The four means describe treated (T) and comparison (C) groups before and after intervention. The contrast subtracts the comparison group’s change from the treated group’s change.
Causal interpretation needs parallel untreated trends and other design assumptions. This expression is not a universal staggered-adoption estimator; specify group-time effects, valid controls and aggregation when adoption varies.
Published example
Examine changes in expansion states relative to comparison states. Evaluate the counterfactual trend argument and policy-level inference rather than treating the comparison as randomized.
This publication illustrates a study design, not an endorsement of every method or a reproduction of its results. Follow its protocol, supplement and update history when studying the example.
Hypothetical scenario
A regional authority wants to evaluate whether a copayment change affected medication adherence, using neighboring regions without the change as comparisons.
- Comparative change overview: State the target population and effect of the policy change.
- Groups and adoption timing: Define adopting and comparison regions, rollout dates and possible spillovers.
- Panel data preparation: Construct comparable region-period outcomes and examine changes in patient composition.
- Identification and estimation plan: Use a timing-compatible estimator, justified covariates and uncertainty reflecting policy-level clustering.
- Diagnostics and sensitivity: Examine pre-policy patterns, anticipation, alternative controls and sensitivity to departures from parallel trends.
Deliver a populated protocol, decision log, data specification, and planned outputs. Enter numerical findings only after analysis and verification. Keep decisions made before seeing results distinguishable from later amendments.
Sources
- Callaway and Sant’Anna: Difference-in-Differences with Multiple Time Periods
- RECORD checklist for studies using routinely collected health data
- HMA–EMA Catalogue of RWD studies: registration and protocol guidance
Source check: 2026-09-17. Record the version of each reporting or appraisal tool used in your actual protocol.
Compare all Observational and RWE studies project types · Open editor in a new tab