Missing Data Using Stata - Statistical Horizons

Missing Data Using StataPaul Allison, Ph.D.Upcoming Seminar:August 15-16, 2017, Stockholm, Sweden

1Missing Data Using Stata2Basics3For Further Reading4Many ptions9Listwise Deletion (Complete Case)10Listwise Deletion (continued)11Listwise Deletion (continued)12Pairwise Deletion (Available Case)13Dummy Variable Adjustment14Imputation15Maximum Likelihood16Properties of Maximum Likelihood17ML with Ignorable Missing Data18ML for 2 x 2 Contingency Table19Maximizing the Likelihood with ℓEM20ML for Quantitative Variables21EM Algorithm22EM for Multivariate Normal Data23EM for Multivariate Normal Data24College Example25Preliminary Analysis 126Preliminary Analysis 227Preliminary Analysis 328EM in Stata29Convert Covariances to Correlations

30EM As Input to regress31Direct ML32Direct ML33Direct ML (cont.)34SEM without Auxiliary Variable35SEM with Auxiliary Variable36SEM Output with Auxiliary Variable37Compare with Listwise Deletion38Regression with Mplus39Regression with Mplus40Logistic Regression with Mplus41Other Capabilities of Mplus42ML for Repeated Measures Data43Binary Example44Estimation in Stata45Figure 146Limitations of Maximum Likelihood47Multiple Imputation48Regression Imputation49Adding a Random Component50Multiple, Random Imputations51Combining the Imputations52Formula for Standard Error53Random Variation in Parameters54Monotonic Missing Data55MI for Monotone Missing Data56Non-Monotone Missing Data57Two Iterative Solutions58MCMC

59MCMC for Multivariate Normal60Software61Steps for MCMC in Stata62MCMC With Stata63Stata Output 164Stata Output 265Formulas66Imputation with the Dependent Variable67Should Missing Data on the Dependent Variable Be Imputed?68How Many Data Sets?69Options for mi impute mvn70Change the Number of Iterations71Change the Prior Distribution72Categorical Variables73Categorical Variables (cont.)74Some Things NOT to Do75Fully Conditional Specification76Logit Imputation of a Binary Variable77Predictive Mean Matching78Fill-In Phase of FCS79Imputation Phase of FCS80Downside of FCS81Software82FCS in Stata for NLSY Data83Impute Output84Estimate Output85Test Output86mi estimate with Other Commands87Multi-Parameter Inference

88Restricted FMI Test89Unrestricted FMI Test90mi test command91Combining Chi-Squares92Stats Not Reported by mi estimate93mibeta for R-square & Standardized94mibeta Output95Interactions and Nonlinearities96Interaction Results97Imputation Model vs. Analysis Model98MI for Panel Data99Hip Fracture Example100Imputing Clustered Data in Stata101Imputation with Cluster Dummies102Imputation in Wide Form103Imputation Via Random Effects104Hip Fracture Example (cont.)105Why Didn’t Imputation Do Better?106Nonignorable Missing Data107Nonignorable Missing Data108Heckman’s Model for Selection Bias109Heckman’s Model in Stata110Heckman’s Model (cont.)111Pattern-Mixture Models with MI112MI for Pattern-Mixture Models113Summary and Review114Summary and Review

Missing Data Using StataPaul D. Allison, Ph.D.February 2016www.StatisticalHorizons.com1BasicsDefinition: Data are missing on some variables forsome observationsProblem: How to do statistical analysis when dataare missing? Three goals: Minimize bias Maximize use of available information Get good estimates of uncertaintyNOT a goal: imputed values “close” to real values.2

For Further ReadingAlso:Allison, Paul D. (2009) "MissingData." Pp. 72-89 in The SAGEHandbook of Quantitative Methodsin Psychology, edited by Roger E.Millsap and Alberto MaydeuOlivares. Thousand Oaks, CA:Sage Publications ds/2012/01/MilsapAllison.pdf3Many Methods Conventional Listwise deletion (complete case analysis)Pairwise deletion (available case analysis)Dummy variable adjustmentImputation Replacement with meansRegressionHot deckNovel Maximum likelihoodMultiple imputationInverse probability weighting (not discussed here)4

AssumptionsMissing completely at random (MCAR)Suppose some data are missing on Y. These data are said to beMCAR if the probability that Y is missing is unrelated to Y orother variables X (where X is a vector of observed variables).Pr (Y is missing X,Y) Pr(Y is missing) MCARis the ideal situation. What variables must be in the X vector? Only variables in themodel of interest. Ifdata are MCAR, complete data subsample is a random samplefrom original target sample. MCARallows for the possibility that missingness on one variablemay be related to missingness on another e.g., sets of variables may always be missing together5AssumptionsMissing at random (MAR)Data on Y are missing at random if the probability that Y ismissing does not depend on the value of Y, after controlling forobserved variablesPr (Y is missing X,Y) Pr(Y is missing X)E.g., the probability of missing income depends on marital status,but within each marital status, the probability of missing incomedoes not depend on income. Considerably weaker assumption than MCAR Only X’s in the model must be considered. But, includingother X’s (correlated with Y) can make MAR more plausible. Can test whether missingness on Y depends on X Cannot test whether missingness on Y depends on Y6

IgnorabilityThe missing data mechanism is said to be ignorable if The data are missing at random andParameters that govern the missing data mechanism aredistinct from parameters to be estimated (unlikely to beviolated)In practice, “MAR” and “ignorable” are used interchangeablyIf MAR but not ignorable (parameters not distinct), methodsassuming ignorability would still be good, just not optimal.If the missing data mechanism is ignorable, there is no needto model it.Any general purpose method for handling missing data mustassume that the missing data mechanism is ignorable.7AssumptionsNot missing at random (NMAR)If the MAR assumption is violated, the missing data mechanismmust be modeled to get good parameter estimates.Heckman’s regression model for sample selection bias is a goodexample.Effective estimation for NMAR missing data requires very goodprior knowledge about missing data mechanism. Data contain no information about what models would be appropriateNo way to test goodness of fit of missing data modelResults often very sensitive to choice of modelListwise deletion able to handle one important kind of NMAR8

Listwise Deletion (Complete Case)Delete any unit with any missing data (onlyuse complete cases)Strengths Easy to implementWorks for any kind of statistical analysisIf data are MCAR, does not introduce any biasin parameter estimatesStandard error estimates are appropriate9Listwise Deletion (continued)Weaknesses May delete a large proportion of cases, resulting in loss ofstatistical powerMay introduce bias if MAR but not MCARRobust to NMAR for predictor variables inregression analysisLet Y be the dependent variable in a regression (any kind) and Xone of the predictors. SupposePr(X is missing X, Y) Pr(X is missing X)Then listwise deletion will not introduce bias.10

Listwise Deletion (continued)Example: Estimate a regression with number of children asdependent variable and income as an independent variable. 30% of cases have missing data on income, persons with high income areless likely to report incomeBut probability of missing income does not depend on number of childrenThen listwise deletion will not introduce any bias into estimates ofregression coefficientsFor logistic regression, listwise deletion is robust to NMAR onindependent OR dependent variable (but not both)Caveat: This property of listwise deletion presumes thatregression coefficients are invariant across subgroups (noomitted interactions).11Pairwise Deletion (Available Case) For linear models, parameters are functions of means,variances and covariances (moments)Estimate each moment with all available nonmissing casesPlug moment estimates into formulas for parametersStrengths: Approximately unbiased if MCARUses all available informationWeaknesses: Standard errors incorrect (no appropriate sample size)Biased if MAR but not MCARMay break down (correlation matrix not positive definite)12

Dummy Variable AdjustmentA popular method for handling missing data on predictors inregression analysis (Cohen and Cohen 1985)In a regression predicting Y, suppose there is missing data on apredictor X.1.2.3.Create a new variable D 1 if X is missing and D 0 if X is present.When X is missing, set X c where c is some constant (e.g., themean of X).Regress Y on both X and D (and any other variables) Produces biased coefficient estimates (Jones, JASA, 1996) So does a related method: For categorical variables, createa separate missing data category But may be appropriate for “doesn’t apply” missing dataMay also be useful for predictive modeling with missingdata.13ImputationAny method that substitutes estimatedvalues for missing values Replacement with meansRegression imputation (replace with conditional means)Problems Often leads to biased parameter estimates (e.g., variances)Usually leads to standard error estimates that are biaseddownward Treats imputed data as real data, ignores inherent uncertaintyin imputed values.14

Maximum LikelihoodChoose as parameter estimates those values which, if true,would maximize the probability of observing what has, in fact,been observed.Likelihood function: Expresses the probability of the data as afunction of the data and the unknown parameter values.Example: Let p(y θ) be the probability density for y, givenθ (a vector of parameters). For a sample of n independentobservations, the likelihood function is15Properties of Maximum LikelihoodTo get ML estimates, we find the value of θthat maximizes the likelihood function.Under usual conditions, ML estimates havethe following properties: Consistent (implies approximately unbiased inlarge samples)Asymptotically efficientAsymptotically normal16

ML with Ignorable Missing DataSuppose we have 2 discrete variables X and Y, and there isignorable missing data on X. Let p(x,y θ) be the jointprobability function.For a single observation with X missing, the likelihood isThe likelihood for the entire sample with m complete cases isThis likelihood may be maximized like any other.17ML for 2 x 2 Contingency TableVoteMaleFemaleYesNo36223752Furthermore, voting wasmissing for 10 males and 15females.The parameters are p11, p12, p21, p22. If we excludecases with missing data, the likelihood is(p11)36(p12)37(p21)22(p22)52If we allow for missing data, the likelihood is(p11)36(p12)37(p21)22(p22)52(p11 p12)10(p21 p22)1518

Maximizing the Likelihood with ℓEMFreeware for Windows by Jeroen manresdimlabsubmoddatOutput2* P(sv) *11 1 0.23802 2 21 2 0.24462 1 0.1538r s v2 2 0.3636sv ssv[36 37 22 52 10 15](0.0339)(0.0342)(0.0297)(0.0384)ℓEM fits a large class of models for categorical data, includinglog-linear, logit, latent class, and discrete time event historymodels.19ML for Quantitative VariablesAssume multivariate normality, which implies All variables are normally distributedAll conditional expectation functions are linearAll conditional variance functions are homoscedasticA strong assumption but widely invoked as the basis formultivariate analysisSeveral ways to get ML estimates with missing data, based onthis assumption Factoring the likelihood for monotone missing data patternsEM algorithmDirect maximization of the likelihood20