Académique Documents
Professionnel Documents
Culture Documents
Graduate Methods Master Class Department of Government, Harvard University February 25, 2005
Data
Observational data
Problem
We cannot observe the counterfactual (what if treatment group had not received treatment) Difference-in-differences
Method
Instrumental variables
The coefficient on the interaction term (3 ) gives us the difference-in-differences estimate of the treatment effect
Difference
1 1 + 3 3
Diff-in-diffs: example
Card and Krueger (1994)
What is the effect of increasing the minimum wage on employment at fast food restaurants? Confounding factor: national recession Treatment group = NJ Control group = PA Before = Feb 92 After = Nov 92
Diff-in-diffs: example
FTEi = 0 + 1 NJi + 2 Nov92i + 3 NJi*Nov92i + e 23.33 -2.89 -2.16 2.75
FTE 23.33 20.44 Control group (PA) 21.17 Treatment group (NJ) 21.03 Time Treatment effect of minimum wage increase = + 2.75 FTE
Diff-in-diff-in-diffs
A difference-in-difference-in-differences (DDD) model allows us to study the effect of treatment on different groups
If we are concerned that our estimated treatment effect might be spurious, a common robustness test is to introduce a comparison group that should not be affected by the treatment For example, if we want to know how welfare reform has affected labor force participation, we can use a DD model that takes advantage of policy variation across states, and then use a DDD model to study how the policy has affected single versus married women
Diff-in-diffs: drawbacks
Diff-in-diff estimation is only appropriate if treatment is random - however, in the social sciences this method is usually applied to data from natural experiments, raising questions about whether treatment is truly random
Also, diff-in-diffs typically use several years of serially-correlated data but ignore the resulting inconsistency of standard errors (see Bertrand, Duflo, and Mullainathan 2004)
If a direct solution (e.g. including the omitted variable) is not available, instrumental variables regression offers an alternative way to obtain a consistent estimator
the part that is uncorrelated with the error (good variation) the part that is correlated with the error (bad variation)
The basic idea behind instrumental variables regression is to isolate the good variation and disregard the bad variation
IV: example
Levitt (1997): what is the effect of increasing the police force on the crime rate?
This is a classic case of simultaneous causality (high crime areas tend to need large police forces) resulting in an incorrectlysigned (positive) coefficient To address this problem, Levitt uses the timing of mayoral and gubernatorial elections as an instrumental variable Is this instrument valid? Relevance: police force increases in election years Exogeneity: election cycles are pre-determined
IV: example
Two-stage least squares:
Stage 1: Decompose police hires into the component that can be predicted by the electoral cycle and the problematic component
policei = 0 + 1 electioni + i
Stage 2: Use the predicted value of policei from the first-stage regression to estimate its effect on crimei
crimei = 0 + 1 police-hati + i
Finding: an increased police force reduces violent crime (but has little effect on property crime)
IV: drawbacks
It can be difficult to find an instrument that is both relevant (not weak) and exogenous
Assessment of instrument exogeneity can be highly subjective when the coefficients are exactly identified IV can be difficult to explain to those who are unfamiliar with it
Sources
Stock and Watson, Introduction to Econometrics
Bertrand, Duflo, and Mullainathan, How Much Should We Trust Differences-in-Differences Estimates? Quarterly Journal of Economics February 2004
Card and Krueger, "Minimum Wages and Employment: A Case Study of the Fast Food Industry in New Jersey and Pennsylvania," American Economic Review, September 1994
Angrist and Krueger, Instrumental Variables and the Search for Identification: From Supply and Demand to Natural Experiments, Journal of Economic Perspectives, Fall 2001 Levitt, Using Electoral Cycles in Police Hiring to Estimate the Effect of Police on Crme, American Economic Review, June 1997