[TE] Treatment Effects - Stata

TitleTreatment effects — Introduction to treatment-effects commandsDescriptionAlso seeDescriptionThis manual documents commands that use observational data to estimate the effect caused bygetting one treatment instead of another. In observational data, treatment assignment is not controlledby those who collect the data; thus some common variables affect treatment assignment and treatmentspecific outcomes. Observational data is sometimes called retrospective data or nonexperimental data,but to avoid confusion, we will always use the term “observational data”.When all the variables that affect both treatment assignment and outcomes are observable, theoutcomes are said to be conditionally independent of the treatment, and the teffects, stteffects,telasso, didregress, and xtdidregress estimators may be used.When not all of these variables common to both treatment assignment and outcomes are observable,the outcomes are not conditionally independent of the treatment, and eteffects, etpoisson, oretregress may be used.teffects and stteffects offer much flexibility in estimators and functional forms for thetreatment-assignment models. teffects provides models for continuous, binary, count, fractional,and nonnegative outcome variables. stteffects provides many functional forms for survival-timeoutcomes. See [TE] teffects intro, [TE] teffects intro advanced, and [TE] stteffects intro for moreinformation. telasso provides models for continuous, binary, count, and nonnegative outcomevariables, and allows for selection of covariates via lasso methods; see [TE] telasso. didregress alsoprovides models for continuous outcomes and is useful when treatment and control groups followparallel trends before the treatment occurs. Beyond this, xtdidregress allows treatment-effectestimation when working with longitudinal or panel data. See [TE] DID intro and [TE] didregressfor information on these commands.eteffects, etpoisson, and etregress offer less flexibility than teffects because more structure must be imposed when conditional independence is not assumed. eteffects is for continuous,binary, count, fractional, and nonnegative outcomes and uses a probit model for binary treatments; see[TE] eteffects. etpoisson is for count outcomes and uses a normal distribution to model treatmentassignment; see [TE] etpoisson. etregress is for linear outcomes and uses a normal distribution tomodel treatment assignment; see [TE] etregress.2

Treatment effects — Introduction to treatment-effects commands3Treatment effects[TE] teffects aipw[TE] teffects ipw[TE] teffects ipwra[TE] teffects nnmatch[TE] teffects psmatch[TE] teffects ra[TE] telasso[TE] didregress[XT] xtdidregressAugmented inverse-probability weightingInverse-probability weightingInverse-probability-weighted regression adjustmentNearest-neighbor matchingPropensity-score matchingRegression adjustmentTreatment-effects estimation using lassoDifference-in-differences estimationFixed-effects difference-in-differences estimationSurvival treatment meSurvival-timeSurvival-timeinverse-probability weightinginverse-probability-weighted regression adjustmentregression adjustmentweighted regression adjustmentEndogenous treatment effects[TE] eteffects[TE] etpoisson[TE] etregressEndogenous treatment-effects estimationPoisson regression with endogenous treatment effectsLinear regression with endogenous treatment effectsTreatment effects with sample selection, endogenous covariates, and random effects[ERM] eregressExtended linear regression[ERM] eintregExtended interval regression[ERM] eprobitExtended probit regression[ERM] eoprobitExtended ordered probit regressionPostestimation tools[TE] tebalance[TE] tebalance box[TE] tebalance density[TE] tebalance overid[TE] tebalance summarize[TE] teoverlap[TE][TE][TE][TE][TE][TE]eteffects postestimationetpoisson postestimationetregress postestimationstteffects postestimationtelasso postestimationdidregress postestimationCheck balance afer teffects or stteffects estimationCovariate balance boxCovariate balance densityTest for covariate balanceCovariate-balance summary statisticsOverlap issonetregressstteffectstelassodidregress and xtdidregress

4Treatment effects — Introduction to treatment-effects commandsAlso see[U] 1.3 What’s new[TE] teffects intro — Introduction to treatment effects for observational data[TE] teffects intro advanced — Advanced introduction to treatment effects for observational data[TE] teffects multivalued — Multivalued treatment effects[TE] stteffects intro — Introduction to treatment effects for observational survival-time data[TE] Glossary

TitleDID intro — Introduction to difference-in-differences estimationDescriptionRemarks and examplesReferencesAlso seeDescriptionDifference in differences (DID) is a method to estimate the average effect of a treatment on thosewho receive the treatment. The method can be applied to two types of observational data: repeatedcross-sections, in which different individuals are observed at different time points, and panel data. Assuch, Stata provides two commands, didregress and xtdidregress. These commands estimatethe average treatment effect on the treated (ATET) of a binary or continuous treatment on a continuousoutcome by fitting a linear model with time fixed effects and group or panel fixed effects.Difference-in-difference-in-differences (DDD) estimation is also available for situations in whichyou want to augment the DID framework to include additional control groups to obtain the ATET.Remarks and examplesRemarks are presented under the following headings:IntroductionIntuition for estimating effectsStandard error considerationsDifferent types of data and specificationSpecifying groups and time as binary indicatorsExcluding group and time effectsConclusionThis entry presents the intuition and some of the technical details for the estimators indidregress and xtdidregress and the diagnostics available after estimation. See [TE] didregressand [TE] didregress postestimation for details on the syntax and worked examples. For a morecomplete discussion and references on the subject, see Angrist and Pischke (2009), Blundell andDias (2009), Imbens and Wooldridge (2009), Lechner (2011), Angrist and Pischke (2015), Abadieand Cattaneo (2018), and Wing, Simon, and Bello-Gomez (2018).IntroductionDID is one of the most venerable causal inference methods used by researchers. DID estimates theeffect of a treatment on the treated group. When thinking about such effects, one usually thinks ofcomparing a group before and after the treatment, perhaps by looking at a graph such as the onebelow:5

DID intro — Introduction to difference-in-differences estimation6.5Outcome of interest77.586200620082010201220142016YearFigure 16.5Outcome of interest77.58A treatment occurred in the year 2010. This might be a government policy, a change in medicinedosage, or any other intervention of interest. We would like to know if the treatment had an effect. Itis clear from the graph that the outcome of interest changed after 2010. Is this due to the treatmentor is something else occurring? Perhaps there are unobserved time effects that affect the treatmentgroup after the treatment. For instance, there could have been a change in weather conditions or aneconomic downfall that affected the treatment group but was not captured in the model or the data.If this is the case, it does not suffice to look at the treatment group before and after the policy. DIDaddresses this by finding a control group, that is, a group that would have been subject to similarunobserved time effects and was not exposed to the treatment. Comparing the treatment group withthe control group before and after the treatment might give us a better understanding of whether thetreatment made a difference. A graph looking at a treatment and control group might look gure 2For both the treatment and the control group, we see that there was a decrease in the mean ofthe outcome after 2010. Therefore, the decrease we saw in the treatment group cannot be attributableentirely to the treatment. (In fact, these are simulated data, and we know the treatment has no effect.)

DID intro — Introduction to difference-in-differences estimation75.56Outcome of interest6.577.58In a DID setup, if the treated group had not received the treatment, we would expect the treated andcontrol groups to experience the same trends. A treatment effect would imply a systematic deviationfrom a common trend that could be observed graphically. This is what we olFigure 3Here both groups experienced a decrease after 2010, but the treatment-group decrease was moresubstantial. The difference in the decreases across groups may indicate the effect of the treatment.Researchers may motivate their analysis with such graphs. However, graphical evidence is notenough. We need statistical validation, so we fit a model.The DID strategy relies on two differences. The first is a difference across time periods. Separatelyfor the treatment group and the control group, we compute the difference of the outcome meanbefore and after the treatment. This across-time difference eliminates time-invariant unobserved groupcharacteristics that confound the effect of the treatment on the treated group. But eliminating groupinvariant unobserved characteristics is not enough to identify an effect. There may be time-varyingunobserved confounders with an effect on the outcome mean, even after we control for time-invariantunobserved group characteristics. Therefore, we incorporate a second difference—a difference betweenthe treatment group and the control group. DID eliminates time-varying confounders by comparingthe treatment group with a control group that is subject to the same time-varying confounders as thetreatment group.The ATET is then consistently estimated by differencing the mean outcome for the treatment andcontrol groups over time to eliminate time-invariant unobserved characteristics and also differencingthe mean outcome of these groups to eliminate time-varying unobserved effects common to bothgroups. These two differences give the method its name and highlight its intuitive appeal. Moreappealing is the fact that you can get the effect of interest, the ATET, from one parameter in a linearregression.When talking about DID, people cite Snow (1849) and Snow (1855) as the first known applications.Snow claimed that cholera was not transmitted by contaminated air or contaminated blood as wasthought by some academics of his time. Snow hypothesized the disease was communicated via waterthat had been polluted with sewage. Below, he describes how he came up with an idea for a naturalexperiment to validate his hypothesis:“In Thomas Street, Horsleydown, there are two courts close together, consisting ofa number of small houses or cottages, inhabited by poor people. The houses occupyone side of each court or alley—the south side of Trusscott’s Court, and the north

8DID intro — Introduction to difference-in-differences estimationside of the other, which is called Surrey Buildings, being placed back to back, with anintervening space, divided into small back areas, in which are situated the privies of boththe courts, communicating with the same drain, and there is an open sewer which passesthe further end of both courts. Now, in Surrey’s buildings the cholera has committedfearful devastation, whilst in the adjoining court there has been but one fatal case, andanother case that ended in recovery. In the former court the slops of dirty water poureddown by the inhabitants into a channel in front of the houses got into the well fromwhich they obtained their water, this being the only difference . . . ”In the first edition (1849) of the text, Snow reports the deaths from cholera from September 23,1848 to August 25, 1849 for five London districts. The number of deaths is higher in the Southand East districts relative to the other three districts, arising from the source of their water supply.Snow obtains a clear motivation for his theory. In the second edition (1855), he collects data beforeand after a pump with contaminated water in Broadstreet, London, is closed. It is then that he cancompare a treated with a control group before and after a treatment to establish a treatment effect.Intuition for estimating effectsWe can build our intuition about the causal inference implied by the DID setup by using thepotential outcomes framework described in [TE] teffects intro and [TE] teffects intro advanced. Weconsider individual-level data for which we sample different individuals at different points in time,a repeated cross-section. The treatment occurs at the group level. For example, the treatment mayoccur at the state, county, or hospital level. All individuals in a given state, county, or hospital eitherare treated or are controls at a given point in time. We index individuals by i, groups by g , and timeby t. We are interested in the effect of a treatment, Digt {0, 1}, on an outcome, Yigt . Suppose thepotential outcome mean of an individual in group g at time t that does not receive the treatment isgiven by the following:E {Yigt (0) g, t} γg γt(1)Above, γg denotes the group effects and γt denotes the time effects. Also suppose the potentialoutcome mean for someone who receives the treatment is given by the following:E {Yigt (1) g, t} γg γt δ(2)The potential outcome framework described above allows us to think of the regression model:Yigt γg γt Dgt δ εigtA regression estimate of δ , the coefficient on the indicator of treatment, consistently estimates theATET in this simplified framework, if we can make standard assumptions needed for linear regressionand for treatment-effects estimators, discussed in [TE] teffects intro advanced, plus one additionalassumption.To introduce this additional assumption, it is sometimes more intuitive to look at a two-period,two-group example. In this case, g {0, 1}, where 0 is the control group and 1 is the treatmentgroup, and t {0, 1}, where 0 is the period before the treatment and 1 occurs after the treatment.To guarantee a consistent estimate of the ATET, we need to make the parallel-trends assumption:E(Yi01 Dgt 1) E(Yi00 Dgt 1) E(Yi01 Dgt 0) E(Yi00 i Dgt 0)

DID intro — Introduction to difference-in-differences estimation9The parallel-trends assumption is stated in terms of the potential outcomes of not being treated, Yi0t ,conditional on treatment, Dgt . It implies that if the treated had not received the treatment, the groupsdefined by Dgt 1 and Dgt 0 would have the same trends. For this to be true, we need groupeffects to be time invariant and time effects to be group invariant. The simple framework describedin (1) and (2) satisfies the parallel-trends assumption.The parallel-trends assumption has a graphical representation. Let’s think again about the casewith multiple time periods. The parallel-trends assumption implies the paths of the outcome variableover time are parallel between the control and treatment groups prior to the date of the treatment. Wecan visually check this assumption by plotting the means of the outcome over time for both groupsor by visualizing the results of the linear trends model. For instance, we might use a graph like theone in figure 2, where we plotted the means over time. After fitting a model using didregress andxtdidregress, you can get both the mean outcome plot and the trends plot by typing. estat trendplotsAnother way to think about the parallel-trends assumption in the pretreatment period is that treatmentand control groups do not change their behavior in anticipation of the treatment. We can think of theparallel-trends assumption as implying that there should be no treatment effect in anticipation of thetreatment. To test this assumption, we could fit a Granger-type causality model where we augmentour model with dummies for each pretreatment–treatment period for the treated observations. A jointtest of the coefficients on these dummies against 0 can be used as a test of the null hypothesis thatno anticipatory effects have taken place. We can perform this test by typing. estat grangerStandard error considerationsWhile a standard linear regression model can be used to estimate the ATET, the best estimate ofthe standard error requires some consideration. Many standard error estimates have been proposed,and each one performs differently depending on the type of DID model being fit and the structure ofthe data. Here we provide a discussion of some of the issues centered on the available standard errorestimates for didregress and xtdidregress. For a more complete discussion, see Cameron andMiller (2015) and MacKinnon (2019) and the references therein.Bertrand, Duflo, and Mullainathan (2004) show that the standard errors for DID estimates areinconsistent if they do not account for the serial correlation of the outcome of interest. Becausethe outcomes studied usually vary at the group and time levels, it makes sense to correct for serialcorrelation. The authors show that using cluster–robust standard errors at the group level wheretreatment occurs provides correct coverage in the presence of serial correlation when the number ofgroups is not too small. Bester, Conley, and Hansen (2011) further show that using cluster–robuststandard errors and using critical values of a t distribution with G 1 degrees of freedom, where Gis the number of groups, is asymptotically valid for a fixed number of groups and a growing samplesize. In other words, you do not need the number of groups to be arbitrarily large, that is, to growasymptotically. Cluster–robust standard errors with G 1 degrees of freedom are the default standarderrors of didregress and xtdidregress.The results of Bertrand, Duflo, and Mullainathan (2004) and Bester, Conley, and Hansen (2011)demonstrate that we could still obtain reliable standard errors even when the number of groups wasnot large. But what about data with a very small number of groups? Several simulation and theoreticalresults suggest that cluster–robust standard errors may still have poor coverage when the number ofgroups is very small or when the number of treated groups is small relative to the number of controlgroups. These scenarios with small group sizes are common in DID studies, and another method ofstandard error estimation should be considered in these situations.

10DID intro — Introduction to difference-in-differences estimationFor cases where the number of groups is small, we provide three alternatives. The first alternativeis to use the wild cluster bootstrap that imposes the null hypothesis that the ATET is 0. Cameron,Gelbach, and Miller (2008) and MacKinnon and Webb (2018) show that the wild cluster bootstrapprovides better inference than using cluster–robust standard errors with t(G 1) critical values. Thesecond alternative comes from Imbens and Kolesár (2016), who show that with a small number ofgroups, you may use bias-corrected standard errors with the degrees of freedom adjustment proposedby Bell and McCaffrey (2002). For the third alternative, you may use aggregation type methods likethose proposed by Donald and Lang (2007); they show that their method works well when the numberof groups is small but the number of individuals in each group is large.When the disparity be

Cross-referencing the documentation When reading this manual, you will find references to other Stata manuals, for example, [U] 27 Overview of Stata estimation commands;[R] regress; and[D] reshape.The first ex-ample is a reference to chapter 27, Overview of Stata estimation commands, in the User's Guide;