Difference in differences: the comparison is between changes

Build the two-by-two reading sheet

Columbia’s methods guide explains comparing changes across intervention and comparison groups. Its parallel-trends assumption concerns how outcomes would have evolved without intervention, not a requirement that the groups start at identical levels. These concepts organize a reading question; they do not validate a particular vaping policy analysis.

Make four labeled cells: intervention before, intervention after, comparison before, and comparison after. Copy only observations reported on the same outcome scale and relevant periods. An empty cell remains empty. The sheet helps expose what comparison the headline used before you examine a model with additional periods or variables.

Columbia University Mailman School of Public Health: Difference-in-Difference Estimation

Practice with fictional attendance counts

This arithmetic example is not a study or a quitting outcome. Imagine a fixed imaginary set of sessions with attendance going from 20 to 28 in one location and from 10 to 15 in another over the same interval. The changes are eight and five; their difference is three attendance counts.

That calculation is different from the final-level difference of thirteen or the first location’s change of eight. None is a causal estimate simply because you can subtract it. The fictional sheet has no justified comparison group or trend assumptions. It illustrates why a number quoted as an intervention effect needs its underlying comparison clearly identified.

Inspect the assumption about changes without intervention

Read the report’s justification for choosing the comparison group and the available earlier trends. A reassuring earlier graph can inform assessment without proving what would have happened afterward. Check the authors’ discussion of changed group composition, other contemporaneous influences, and whether effects could spill into the comparison group.

Different starting levels do not automatically disqualify the design; different unexplained trajectories can raise a more pertinent question. Avoid inventing one simple pass threshold from a visual inspection. Modern analyses can have additional complications, such as different intervention start times. This guide does not select an estimator or decide whether a named study’s assumptions hold.

Keep the comparison when you state the finding

Your note can identify that the reported estimate compares these groups’ changes under the stated assumptions. If all you have is a single-group before-after result, do not call it difference in differences. If the comparison is documented but its justification remains unclear, name the unanswered trend question instead of declaring the result settled.

For care decisions, ask a qualified professional about relevant evidence rather than deriving a personal probability from a policy estimate. Your own tracking record is one observed trajectory, not the comparison group that would establish what happened because of an intervention. The useful output is the correct change comparison and a focused question about the unobserved alternative trend.