Introduction To Nonparametric/Semiparametric

IntroductionBandwidth Selection for Estimation of DensitiesLocal Linear RegressionRegression Discontinuity DesignIntroduction to Nonparametric/SemiparametricEconometric Analysis: ImplementationYoichi AraiNational Graduate Institute for Policy Studies2014 JEA Spring Meeting (14 June)1 / 30

IntroductionBandwidth Selection for Estimation of DensitiesLocal Linear RegressionRegression Discontinuity DesignIntroductionMotivationMSE (MISE): Measures of DiscrepancyChoice of Kernel FunctionsBandwidth Selection for Estimation of DensitiesBandwidth Selection I: Rule of Thumb BandwidthBandwidth Selection II: Plug-In MethodBandwidth Selection III: Cross ValidationLocal Linear RegressionBandwidth Selection I: Plug-In MethodBandwidth Selection II: Cross-ValidationBandwidth Selection III: More Sophisticated MethodRegression Discontinuity DesignRegression Discontinuity DesignBandwidth Selection2 / 30

IntroductionBandwidth Selection for Estimation of DensitiesLocal Linear RegressionRegression Discontinuity DesignMotivationMSE (MISE): Measures of DiscrepancyChoice of Kernel FunctionsMotivation: RD Estimates of the Effect of Head StartAssistance by Ludwig and Miller (2007, QJE)VariableBandwidthNumber of obs. with nonzero weight1968 HS spending per childRD estimate1972 HS spending per childRD estimateAge 5–9, Mortality, 1973–83RD estimateBlacks age 5–9, Mortality, 1973–83RD estimate9[217, 310]Nonparametric1836[287, 674] [300, 1877]137.251(128.968)114.711(91.267)134.491 (62.593)182.119 (148.321)88.959(101.697)130.153 (67.613) 1.895 (0.980) 1.198 (0.796) 1.114 (0.544) 2.275(3.758) 2.719 (2.163) 1.589(1.706)3 / 30

IntroductionBandwidth Selection for Estimation of DensitiesLocal Linear RegressionRegression Discontinuity DesignMotivationMSE (MISE): Measures of DiscrepancyChoice of Kernel FunctionsObservations Estimates can change dramatically by the choice of bandwidths. Statistical significance can also change depending on the choice ofbandwidths.Lessons It would be nice to have objective criterion to choose bandwidths!4 / 30

IntroductionBandwidth Selection for Estimation of DensitiesLocal Linear RegressionRegression Discontinuity DesignMotivationMSE (MISE): Measures of DiscrepancyChoice of Kernel FunctionsMSE (MISE): Measures of DiscrepancySuppose your objective is to estimate some function f (x) (f evaluated at x), or some function f over entire support.Let fˆh be some estimator based on a bandwidth h.Most Popular Measures of Discrepancy of fˆ from the true objective f MSE (x) E [{fˆh (x) f (x)}2 ] (Local Measure).MISE E [{fˆh (x) f (x)}2 ]dx (Global Measure).MSE and MISE changes depending on the function f as well asestimation methods.5 / 30

IntroductionBandwidth Selection for Estimation of DensitiesLocal Linear RegressionRegression Discontinuity DesignMotivationMSE (MISE): Measures of DiscrepancyChoice of Kernel FunctionsKernel FunctionsTable : Popular Choices for Kernel nction2(2π) 1/2 e x /2(1/2)1{ x 1}(3/4)(1 x 2 )1{ x 1}(1 x )1{ x 1}Practical Choices It is well-known that nonparametric estimates are not very sensitiveto the choice of kernel functions. For estimating a function at interior points or globally, a commonchoice is the Epanechnikov kernel (Hodges & Lehmann, 1956). For estimating a function at boundary points (by LLR), a popularchoice is that the triangular kernel (Cheng, Fan & Marron,1997).6 / 30

IntroductionBandwidth Selection for Estimation of DensitiesLocal Linear RegressionRegression Discontinuity DesignMotivationMSE (MISE): Measures of DiscrepancyChoice of Kernel FunctionsBandwidth SelectionContrary to the selection of kernel functions, it is well-known thatestimates are sensitive to the choice of bandwidths.In the following, we briefly explain 3 popular approaches for bandwidthselection1. Rule of Thumb Bandwidth2. Plug-In Method3. Cross-ValidationSee Silverman (1986) for more about basic treatment on densityestimation.7 / 30

IntroductionBandwidth Selection for Estimation of DensitiesLocal Linear RegressionRegression Discontinuity DesignBandwidth Selection I: Rule of Thumb BandwidthBandwidth Selection II: Plug-In MethodBandwidth Selection III: Cross ValidationAMSE for Kernel Density EstimatorsGiven a random sample {Xi , i 1, 2, . . . , n}, we are interested inestimating its density f .For the kernel density estimator1 nx Xifˆh ), K (nh i 1hthe asymptotic approximation of the MSE (AMSE) is given byAMSE (x) {2µ2 (2)κ2 f (x)f (x)h2 } 2nhwhere f (r ) is the r -th derivative of f .Similarly, the asymptotic approximation of the MISE (AMISE) is given byAMISE µ22κ2{ f (2) (x)2 dx} h4 4nh8 / 30

IntroductionBandwidth Selection for Estimation of DensitiesLocal Linear RegressionRegression Discontinuity DesignBandwidth Selection I: Rule of Thumb BandwidthBandwidth Selection II: Plug-In MethodBandwidth Selection III: Cross ValidationOptimal BandwidthBandwidths that minimize the AMSE and AMISE are given, respectively,by1/5f (x)hAMSE C (K ) { (2) 2 } n 1/5f (x)and1/5hAMISE C (K )²depends on1} n 1/5(2) f (x)2 dxK ¹¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹¶depends on f{where C (K ) {κ2 /µ22 }1/5 . Both hAMSE and hAMISE depend on 3 things1. K (Kernel function),2. f (true density including the 2nd derivative f (2) ),3. n (sample size).In addition, hAMSE depends on the evaluation point x.9 / 30

IntroductionBandwidth Selection for Estimation of DensitiesLocal Linear RegressionRegression Discontinuity DesignBandwidth Selection I: Rule of Thumb BandwidthBandwidth Selection II: Plug-In MethodBandwidth Selection III: Cross ValidationBandwidth Selection I: Rule of Thumb BandwidthRule of Thumb (ROT) Bandwidth can be obtained by specifying, forhAMISE , Gaussian kernel for K , and Gaussian density with variance σ 2 for f ,implyinghROT 1.06σn 1/5 .Remark In practice, we use an estimated σ̂ for σ. This is the default bandwidth used by Stata command kdensity.Obviously, hROT works well if the true density is Gaussian. Not necessarily works well if the true density is not Gaussian.10 / 30

IntroductionBandwidth Selection for Estimation of DensitiesLocal Linear RegressionRegression Discontinuity DesignBandwidth Selection I: Rule of Thumb BandwidthBandwidth Selection II: Plug-In MethodBandwidth Selection III: Cross ValidationBandwidth Selection II: Plug-In MethodRather than assuming Gaussian density, the plug-in method estimates f and f (2) for hAMSE , ψ f 2 (x)2 dx for hAMISE .A standard kernel density and density derivative estimator is given by1 nx Xifˆa1 (x) ), K (na1 i 1a1fˆa(d)(x) 2n1(d) x Xi) K (d 1a2na2 i 1ψ can be estimated bynψ̂ n 1 fˆa(4)(Xi ).3i 1Remark These require to choose the bandwidths a1 , a2 and a3 .Those are usually chosen by a simple rule such as the ROT rule.The plug-in method introduced here is often called direct plug-in(DPI).11 / 30

IntroductionBandwidth Selection for Estimation of DensitiesLocal Linear RegressionRegression Discontinuity DesignBandwidth Selection I: Rule of Thumb BandwidthBandwidth Selection II: Plug-In MethodBandwidth Selection III: Cross ValidationBandwidth Selection II: Plug-In MethodThere exists a more sophisticated method proposed by Sheather andJones (1991, JRSS B). The pilot bandwidths such as a1 , a2 , a3 can be written as a functionof h. Determine the bandwidths h and the pilot bandwidthssimultaneously.The bandwidth chosen in this manner is called the solve-the-equation(STE) rule.Remark Simulation studies show the STE bandwidths perform very well. The DPI and STE bandwidths can be obtained by the Statacommand kdens. See also Wand and Jones (1994) for more about these bandwidths.12 / 30

IntroductionBandwidth Selection for Estimation of DensitiesLocal Linear RegressionRegression Discontinuity DesignBandwidth Selection I: Rule of Thumb BandwidthBandwidth Selection II: Plug-In MethodBandwidth Selection III: Cross ValidationBandwidth Selection III: Cross ValidationLeast Squares Cross Validation (LSCV) bandwidth minimizesnLSCV (h) fˆh (x)2 dx 2n 1 fˆ i,h (Xi )i 1where the leave-one-out kernel density estimator is given byfˆ i,h (x) 1 n x Xj). (n 1 j ihRationale for the LSCV Observe that22 (fˆh (x) f (x)) dx R(fˆh ) f (x) dx.whereR(fˆh ) fˆh (x)2 dx 2 fˆh (x)f (x)dx. Then we can show thatE [LSCV (h)] E [R(fˆ)].13 / 30

IntroductionBandwidth Selection for Estimation of DensitiesLocal Linear RegressionRegression Discontinuity DesignBandwidth Selection I: Rule of Thumb BandwidthBandwidth Selection II: Plug-In MethodBandwidth Selection III: Cross ValidationBandwidth Selection III: Cross ValidationSome Remarks on the LSCV The LSCV is based on the global measure by construction. The LSCV requires numerical optimization. Then the LSCV can be computationally very intensive. Some simulation studies show that the LSCV bandwidth tends to bevery dispersed.14 / 30

IntroductionBandwidth Selection for Estimation of DensitiesLocal Linear RegressionRegression Discontinuity DesignBandwidth Selection I: Plug-In MethodBandwidth Selection II: Cross-ValidationBandwidth Selection III: More Sophisticated MethodAMSE for the Local Linear RegressionGiven a random sample {(Yi , Xi ), i 1, 2, . . . , n}, we are interested inestimating the regression functionm(x) E [Yi Xi x].The local linear regression can be obtained by minimizingn2 {yi α β(Xi x)} K (i 1Xi x)hand the resulting α̂ estimates m(x).The AMSE for the LLR is given byAMSE (x) µ22 (2)κ2 σ 2 (x)m (x)h4 4nhf (x)LLR is popular because of design adaptation property especially atboundary points. (See Fan and Gijbels, 1996.)15 / 30

IntroductionBandwidth Selection for Estimation of DensitiesLocal Linear RegressionRegression Discontinuity DesignBandwidth Selection I: Plug-In MethodBandwidth Selection II: Cross-ValidationBandwidth Selection III: More Sophisticated MethodOptimal Bandwidth for the Local Linear RegressionThe optimal bandwidth is given by1/5hAMSE C (K ) {σ 2 (x)}m(2) (x)2 f (x)n 1/5 .For global estimation, the commonly used bandwidth minimizes AMSE (x)w (x)dxwhere w (x) is a weighting function and it is given by1/5hAMISEσ 2 (x)w (x)/f (x)dx C (K ){ } n 1/5 . m(2) (x)2 w (x)dx²depends on K ¹¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¹ ¶depends on m(2) ,σ2 ,f , and w16 / 30

IntroductionBandwidth Selection for Estimation of DensitiesLocal Linear RegressionRegression Discontinuity DesignBandwidth Selection I: Plug-In MethodBandwidth Selection II: Cross-ValidationBandwidth Selection III: More Sophisticated MethodBandwidth Selection I: Plug-In MethodThe plug-in bandwidth is given by1/5hROT C (K ) {σ̂ 2 w (x)dx}n i 1 m̂(2) (Xi )2 w (Xi ).where σ̂ 2 and m̂(2) are obtained by the global polynomial regression oforder 4.Remark: A possible choice for w (x) is the uniform kernel constructed tocover 90% of the sample. This is the default bandwidth used by the Stata command lpoly. This bandwidth is also called the ROT bandwidth.17 / 30

IntroductionBandwidth Selection for Estimation of DensitiesLocal Linear RegressionRegression Discontinuity DesignBandwidth Selection I: Plug-In MethodBandwidth Selection II: Cross-ValidationBandwidth Selection III: More Sophisticated MethodBandwidth Selection II: Cross-ValidationThe bandwidth based on the cross-validation minimizesnCV (h) {yi fˆ i,h (Xi )}2i 1where fˆ i, is the leave-one-out LLR estimates.That ishCV arg min CV (h)hRemark This bandwidth can be obtained by the Stata command locreg18 / 30