BOOTSTRAP METHODS FOR TIME SERIES
BOOTSTRAP METHODS FOR TIME SERIESbyWolfgang HärdleInstitute for Statistics and EconometricsHumboldt Universität zu BerlinBerlin, GERMANYJoel HorowitzDepartment of EconomicsNorthwestern UniversityEvanston, ILJens-Peter KreissInstitute for Mathematical StochasticsTechnical University of BraunschweigBraunschweig, GERMANYAugust 2001AbstractThe bootstrap is a method for estimating the distribution of an estimator or test statisticby resampling one’s data or a model estimated from the data. The methods that are available forimplementing the bootstrap and the accuracy of bootstrap estimates depend on whether the dataare a random sample from a distribution or a time series. This paper is concerned with theapplication of the bootstrap to time-series data when one does not have a finite-dimensionalparametric model that reduces the data generation process to independent random sampling. Wereview the methods that have been proposed for implementing the bootstrap in this situation anddiscuss the accuracy of these methods relative to that of first-order asymptotic approximations.We argue that methods for implementing the bootstrap with time-series data are not as wellunderstood as methods for data that are sampled randomly from a distribution. Moreover, theperformance of the bootstrap as measured by the rate of convergence of estimation errors tends tobe poorer with time series than with random samples. This is an important problem for appliedresearch because first-order asymptotic approximations are often inaccurate and misleading withtime-series data and samples of the sizes encountered in applications. We conclude that there is aneed for further research in the application of the bootstrap to time series, and we describe someof the important unsolved problems.The research of Wolfgang Härdle and Jens-Peter Kreiss was supported in part by DeutscheForschungsgemeinschaft Sonderforschungsbereich 373, “Quantifikation und SimulationÖkonomischer Prozesse.” The research of Joel L. Horowitz was supported in part by NSF GrantSES-9910925 and by the Alexander von Humboldt Foundation.
BOOTSTRAP METHODS FOR TIME SERIES1. IntroductionThe bootstrap is a method for estimating the distribution of an estimator or test statisticby resampling one’s data or a model estimated from the data. Under conditions that hold in awide variety of applications, the bootstrap provides approximations to distributions of statistics,coverage probabilities of confidence intervals, and rejection probabilities of tests that are at leastas accurate as the approximations of first-order asymptotic distribution theory.Often, thebootstrap provides approximations that are more accurate than those of first-order asymptotictheory.The methods that are available for implementing the bootstrap and the improvements inaccuracy that it achieves relative to first-order asymptotic approximations depend on whether thedata are a random sample from a distribution or a time series. If the data are a random sample,then the bootstrap can be implemented by sampling the data randomly with replacement or bysampling a parametric model of the distribution of the data. The distribution of a statistic isestimated by its empirical distribution under sampling from the data or parametric model. Beranand Ducharme (1991), Hall (1992), Efron and Tibshirani (1993), and Davison and Hinkley (1997)provide detailed discussions of bootstrap methods and their properties for data that are sampledrandomly from a distribution.The situation is more complicated when the data are a time series because bootstrapsampling must be carried out in a way that suitably captures the dependence structure of the datageneration process (DGP). This is not difficult if one has a finite-dimensional parametric model(e.g., a finite-order ARMA model) that reduces the DGP to independent random sampling. Inthis case and under suitable regularity conditions, the bootstrap has properties that are essentiallythe same as they are when the data are a random sample from a distribution. See, for example,Andrews (1999) and Bose (1988, 1990).This paper is concerned with the situation in which one does not have a finitedimensional parametric model that reduces the DGP to independent random sampling. Wereview the methods that have been proposed for implementing the bootstrap in this situation anddiscuss the ability of these methods to achieve asymptotic refinements. We argue that methodsfor implementing the bootstrap with time-series data are not as well understood as methods fordata that are sampled randomly from a distribution. Moreover, the performance of the bootstrapas measured by the order of the asymptotic refinements that are available from known methodstends to be poorer with time series than with random samples. This is an important problem for1
applied research because first-order asymptotic approximations are often inaccurate andmisleading with time-series data and samples of the sizes encountered in applications. Weconclude that there is a need for further research in the application of the bootstrap to time series,and we describe some of the important unsolved problems.Section 2 of this paper describes the estimation and inference problems that will bediscussed in the remainder of the paper. Section 2 also provides background information on theperformance of the bootstrap when the data are a random sample from a distribution and on thetheory underlying the bootstrap’s ability to provide asymptotic refinements. Section 3 reviewsthe block bootstrap, which is the oldest and best known nonparametric method for implementingthe bootstrap with time-series data.The block bootstrap imposes relatively few a priorirestrictions on the DGP, but this flexibility comes at the price of estimation errors that convergeto zero relatively slowly. Section 4 discusses methods that make stronger assumptions about theDGP but offer the possibility of faster converging estimation errors.Section 5 presentsconclusions and suggestions for further research. The regularity conditions required by bootstrapmethods for time-series tend to be highly technical, and they vary among investigators andmethods for implementing the bootstrap. To enable us to concentrate on important ideas ratherthan technicalities, we do not give detailed regularity conditions in this paper. They are availablein the references that are cited.We assume throughout this paper that the DGP is stationary and weakly dependent.Bootstrap methods for DPG’s that do not satisfy this condition, notably long-memory and unitroot processes, are important topics for research but are at a much more preliminary stage ofdevelopment than are methods for stationary, weakly dependent processes.2. Problem Definition and Background InformationThis section has three parts. Section 2.1 describes the estimation and inference problemsthat will be treated in the remainder of the paper. Section 2.2 reviews the performance of thebootstrap when the data are a random sample from a distribution. This performance provides auseful benchmark for judging the bootstrap’s performance when the data are a time series.Section 2.3 reviews the theory underlying the bootstrap’s ability to provide asymptoticrefinements.2
2.1 Statement of the ProblemLet { X i : i 1,., n} be observations from the sequence { X i : i } , whereX i \ d for each integer i and some integer d satisfying 1 d . Unless otherwise stated, weassume that { X i } is a realization of a discrete-time stochastic process (the DGP) that is strictlystationary and geometrically strongly mixing (GSM). We also assume that µ E ( X1 ) exists.Define mn n 1 i 1 X i .nLet θ : \ d \ be a function. In this paper, we are concerned withmaking inferences about θ θ ( µ ) based on the estimator θ n θ (mn ) .As is discussed by Hall and Horowitz (1996) and Andrews (2002), a wide variety ofestimators that are important in applications can be approximated with (higher-order)asymptotically negligible errors by functions of the form θ (mn ) . Thus, the focus on estimatorsof this form is not highly restrictive. In particular, generalized-method-of-moments (GMM)estimators can be approximated this way under mild regularity conditions (Hall and Horowitz1996, Andrews 2001). GMM is a method for estimating a possibly vector-valued parameter ψthat is identified by the moment condition Eg ( X ,ψ ) 0 , where g is a function whosedimension equals or exceeds that of ψ . The class of GMM estimators includes linear andnonlinear least squares estimators and maximum likelihood estimators, among many others.Hansen (1982) provides details of the GMM estimation method and gives conditions under whichGMM estimators are n1/ 2 -consistent and asymptotically normal.Assume now that Eθ (mn ) and σ θ2 Var[θ (mn )] exist. Define σ 2 to be the variance ofthe asymptotic distribution of n1/ 2 (θ n θ ) , and let sn2 be a weakly consistent estimator of σ 2 .In the remainder of this paper, we discuss the use of the bootstrap to estimate the followingquantities, all of which are important in applied research:1.The bias and variance of θ n , that is Eθ n θ and σ θ2 .2.Theone-sideddistributionfunctionsP[n1/ 2 (θ n θ ) τ ] ,P[n1/ 2 (θ n θ ) / σ τ ] , and P[n1/ 2 (θ n θ ) / sn τ ] for any real τ .3.ThesymmetricaldistributionP[n1/ 2 (θ n θ ) / σ τ ] , and P[n1/ 2 (θ n θ ) / sn τ ] .3functionsP[n1/ 2 (θ n θ ) τ ] ,
4.The coverage probabilities of one-sided and symmetrical confidence intervals forθ . These are θ θ n zˆ1 α sn , θ n zˆα sn θ , and θ n zˆα / 2 sn θ θ n zˆα / 2 sn , whereẑα is a bootstrap estimator of the 1 α quantile of the distribution of n1/ 2 (θ n θ ) / sn .5.The probabilities that one-sided and symmetrical tests reject the correct nullhypothesis H 0 : θ θ 0 . For one-sided tests, the test statistic is n1/ 2 (θ n θ 0 ) / sn with bootstrapcritical values ẑα and ẑ1 α for upper- and lower- tail tests, respectively. For a symmetrical test,the test statistic is n1/ 2 (θ n θ ) / sn , and the bootstrap critical value is ẑα / 2 .The conclusions regarding coverage probabilities of confidence intervals and rejectionprobabilities of tests are identical, so only the rejection probabilities of tests are treated explicitlyhere.2.2 Performance of the Bootstrap when the Data Are a Random SampleThe rates of convergence of the errors made by first-order asymptotic approximations andby bootstrap estimators with data that are a random sample provide useful benchmarks forjudging the bootstrap’s performance with time series. As will be discussed in Sections 3 and 4,the errors made by the bootstrap converge to zero more slowly when the data are a time seriesthan when they are a random sample. In some cases, the rates of convergence for time series dataare close to those for a random sample, but in others they are only slightly faster than the rateswith first-order approximations.If the data are a random sample, then under regularity conditions that are given by Hall(1992),1.The bootstrap estimates σ θ2 consistently and reduces the bias of θ n to O( n 2 ) .That is E (θ n θ Bˆn ) O( n 2 ) , where Bˆ n is the bootstrap estimator of E (θ n θ ) . By contrast,E (θ n θ ) O( n 1 ) . Horowitz (2001a) gives an algorithm for computing Bˆ n .If, in addition, n1/ 2 (θ n θ ) / sn d N (0,1) , then:2.The errors in the bootstrap estimates of the one-sided distribution functionsP[n1/ 2 (θ n θ ) / σ τ ] and P[n1/ 2 (θ n θ ) / sn τ ] are O p (n 1 ) . The errors made by first orderasymptotic approximations are O(n 1/ 2 ) .By “error” we mean the difference between abootstrap estimator and the population probability that it estimates.4
3.The errors in the bootstrap estimates of the symmetrical distribution functionsP[n1/ 2 (θ n θ ) / σ τ ] and P[n1/ 2 (θ n θ ) / sn τ ] are O p (n 3 / 2 ) , whereas the errorsmade by first-order approximations are O (n 1 ) .4.When the bootstrap is used to obtain the critical value of a one-sided hypothesistest, the resulting difference between the true and nominal rejection probabilities under the nullhypothesis (error in the rejection probability or ERP) is O (n 1 ) , whereas it is O(n 1/ 2 ) when thecritical value is obtained from first-order approximations. The same result applies to the error inthe coverage probability (ECP) of a one-sided confidence interval. In some cases, the bootstrapcan reduce the ERP (ECP) of a one-sided test (confidence interval) to O( n 3 / 2 ) (Hall 1992, p.178; Davidson and MacKinnon 1999).5.When the bootstrap is used to obtain the critical value of a symmetricalhypothesis test, the resulting ERP is O( n 2 ) , whereas it is O (n 1 ) when the critical value isobtained from first-order approximations. The same result applies to the ECP of a symmetricalconfidence interval.2.3 Why the Bootstrap Provides Asymptotic RefinementsThis section outlines the theory underlying the bootstrap’s ability to provide asymptoticrefinements.To minimize the length of the discussion, we concentrate on the distributionfunction of the asymptotically N(0,1) statistic Tn n1/ 2 (θ n θ ) / sn and the ERP of a symmetricalhypothesis test based on this statistic.Similar arguments apply to one-sided tests and toconfidence intervals. Hall (1992) gives regularity conditions for the results of this section whenthe data that are a random sample from a distribution. The references cited in Sections 3-4 giveregularity conditions for time series.Let P̂ denote the probability measure induced by bootstrap sampling, and let Tˆn denotea bootstrap analog of Tn . If the data are a random sample from a population, then it suffices tolet P̂ be the empirical distribution of the data. Bootstrap samples are then drawn by samplingthe data { X i : i 1,., n} randomly with replacement. If { Xˆ i : i 1,., n} is such a sample, thenTˆn n1/ 2 (θˆn θ n ) / sˆn ,where θˆn θ (mˆ n ) , mˆ n n 1formula for sn2 . i 1 Xˆ i , and sˆn2 is obtained by replacing the {X i } with {Xˆ i } in thenFor example, if Σ n denotes the sample covariance matrix of X and5
sn2 θ (mn )′Σ n θ (mn ) , then sˆn2 θ (mˆ n )′Σˆ n θ (mˆ n ) , where Σˆ n is the sample covariancematrix of the bootstrap sample. Bootstrap versions of Tˆn for time-series data are presented inSections 3-4. The discussion in this section does not depend on or require knowledge of thedetails of the bootstrap versions of Tˆn .The arguments showing that the bootstrap yields asymptotic refinements are based onEdgeworth expansions of P (Tn z ) and Pˆ(Tˆn z ) . Additional notation is needed to describethe expansions. Let Φ and φ , respectively, denote the standard normal distribution function anddensity.The j’th cumulant of Tn ( j 4) has the form n 1/ 2κ j o(n 1/ 2 ) if j is odd andI ( j 2) n 1κ j o(n 1 ) if j is even, where κ j is a constant and I is the indicator function(Hall 1992, p. 46). Define κ (κ1 ,.,κ 4 )′ . Conditional on the data { X i : i 1,., n} , the j’thcumulant of Tˆnalmost surely has the formI ( j 2) n 1κˆ j o(n 1 ) if j is even.n 1/ 2κˆ j o(n 1/ 2 )if j is odd andThe quantities κˆ j depend on { X i } .They arenonstochastic relative to bootstrap sampling but are random variables relative to the stochasticprocess that generates { X i } . Define κˆ (κˆ1, .,κˆ 4)′ .P (Tn z ) has the Edgeworth expansion(2.1)P (Tn z ) Φ ( z ) 2 n j / 2 g j ( z,κ )φ ( z) O(n 3 / 2 )j 1uniformly over z, where g j ( z ,κ ) is a polynomial function of z for each κ , a polynomial functionof the components of κ for each z , an even function of z if j 1, and an odd function of z if j 2. Moreover, P ( Tn z ) has the expansion(2.2)P ( Tn z ) 2Φ ( z ) 1 2n 1 g 2 ( z ,κ )φ ( z ) O(n 2 )uniformly over z. Conditional on the data, the bootstrap probabilities Pˆ(Tˆn z ) and Pˆ( Tˆn z )have the expansions(2.3)Pˆ(Tˆn z ) Φ ( z ) 2 n j / 2 g j ( z,κˆ )φ ( z) O(n 3 / 2 )j 1and(2.4)Pˆ ( Tˆn z ) 2Φ ( z ) 1 2n 1 g 2 ( z ,κˆ )φ ( z ) O(n 2 )6
uniformly over z almost surely. Let κ g j denote the gradient of g j with respect to its secondargument. Then a Taylor series expansion yields(2.5)( Pˆ(Tˆn z ) P (Tn z ) n 1/ 2 [ κ g1 ( z ,κ )(κˆ κ )]φ ( z ) O n 1/ 2 κˆ κ2) O(n 1)and(2.6)( Pˆ( Tˆn z ) P ( Tn z ) 2n 1 [ κ g 2 ( z ,κ )(κˆ κ )]φ ( z ) O n 1 κˆ κalmost surely uniformly over z.2) O(n 2)Thus, the leading terms of the errors made by bootstrapestimators of one-sided and symmetrical distribution functions are n 1/ 2 [ κ g1 ( z ,κ )(κˆ κ )]φ ( z )and 2n 1 [ κ g 2 ( z ,κ )(κˆ κ ) ]φ ( z ) , respectively.If the data are a random sample from adistribution, then κˆ κ O p (n 1/ 2 ) , so the errors of the bootstrap estimators are O p (n 1 ) andO p (n 3 / 2 ) for one-sided and symmetrical distribution functions, respectively. The root-meansquare estimation errors (RMSE’s) also converge at these rates. As will be discussed in Sections3 and 4, the rate of convergence of κˆ κ is slower than n 1/ 2 when the data are a time series.Thus, the errors of bootstrap estimators of distribution functions are larger when the data are atime series than when they are a random sample.Now consider the ERP of a symmetrical hypothesis test.Let zα / 2 and ẑα / 2 ,respectively, denote the 1 α / 2 quantiles of the distributions of Tn and Tˆn .ThenP ( Tn zα / 2 ) Pˆ ( Tˆn zˆα / 2 ) 1 α . The bootstrap-based symmetrical test at the nominal αlevel accepts H 0 if Tn ẑα / 2 . Thus, the ERP of the test is P ( Tn zˆα / 2 ) (1 α ) .To derive the ERP, use (2.1) and (2.3) to obtain(2.7)2Φ ( zα ) 1 2n 1 g 2 ( zα ,κ )φ ( zα ) 1 α O (n 2 )and(2.8)2Φ ( zˆα ) 1 2n 1 g 2 ( zˆα ,κˆ )φ ( zα ) 1 α O (n 2 )almost surely. Let vα denote the 1 α / 2 quantile of the N (0,1) distribution. Then CornishFisher inversions of (2.7) and (2.8) giveznα vα n 1 g 2 (vα ,κ ) O (n 2 )andzˆnα vα n 1 g 2 (vα ,κˆ ) O (n 2 )almost surely (Hall 1992). Therefore,7
(2.9)P ( Tn zˆnα) P{ Tn znα n 1[ g 2 (vα ,κˆ ) g 2 (vα ,κ )] O (n 2 )} .Now suppose that κˆ κ O (n a ) almost surely for some a 0 . Since the components of κcan be estimated with errors that are no smaller than O p (n 1/ 2 ) , it follows from (2.9) thatP ( Tn zˆα / 2 ) 1 α O(n 1 a ) .Thus, in general, the rate of convergence of the ERP (and of the ECP for a symmetricalconfidence interval) is determined by the rate of convergence of κˆ κ .If the data are a random sample from a distribution, then it is possible to carry out anEdgeworth expansion of the right-hand side of (2.9). This yields(2.10)P ( Tn zˆα / 2 ) 1 α O( n 2 )See Hall (1992, pp. 108-114).Thus, the ERP of a symmetrical test (and the ECP of asymmetrical confidence interval) based on the bootstrap critical value ẑα / 2 is O( n 2 ) when thedata are a random sample from a population. As will be discussed in Sections 3 and 4, this rate ofthe ERP is not available with current bootstrap methods for time series. Rather, the ERP fortime-series data is O( n 1 a ) for some a satisfying 0 a 1/ 2 .3. The Block BootstrapThe block bootstrap is the best-known method for implementing the bootstrap with timeseries data. It consists of dividing the data into blocks of observations and sampling the blocksrandomly with replacement. The blocks may be non-overlapping (Hall 1985, Carlstein 1986) oroverlapping (Hall 1985, Künsch 1989, Politis and Romano 1993). To describe these blockingmethods more precisely, let the data consist of observations { X i : i 1,., n} . With non-overlappingblocks of length A , block 1 is observations { X j : j 1,., A} , block 2 is observations{ X A j : j 1,., A} , and so forth. With overlapping blocks of length A , block 1 is observations{ X j : j 1,., A} , block 2 is observations { X j 1 : j 1,., A} , and so forth. The bootstrap sample isobtained by sampling blocks randomly with replacement and laying them end-to-end in the ordersampled. It is also possible to use overlapping blocks with lengths that are sampled randomly fromthe geometric distribution (Politis and Romano 1993). The block bootstrap with random blocklengths is also called the stationary bootstrap because the resulting bootstrap data series isstationary, whereas it is not with overlapping or non-overlapping blocks of non-stochastic lengths.8
Regardless of the blocking method that is used, the block length (or average block length inthe stationary bootstrap) must increase with increasing sample size n to make bootstrap estimatorsof moments and distribution functions consistent (Carlstein 1986, Künsch 1989, Hall et al. 1995).Similarly, the block length must increase with increasing sample size to enable the block bootstrapto achieve asymptotically correct coverage probabilities for confidence intervals and rejectionprobabilities for hypothesis tests. When the objective is to estimate a moment or distributionfunction, the asymptotically optimal block length may be defined as the one that minimizes theasymptotic mean-square error of the block bootstrap estimator. When the objective is to form aconfidence interval or test a hypothesis, the asymptotically optimal block length may be defined asthe one that minimizes the ECP of the confidence interval or ERP or the test. The asymptoticallyoptimal block length and the corresponding rates of convergence of block bootstrap estimationerrors, ECP’s and ERP’s depend on what is being estimated (e.g., bias, a one-sided distributionfunction, a symmetrical distribution function, etc.). The optimal block lengths and the rates ofconvergence of block bootstrap estimation errors with non-stochastic block lengths are discussed indetail in Section 3.2. The accuracy of the stationary bootstrap is discussed in Section 3.3. Theperformance of some modified forms of the block bootstrap are discussed in Sections 3.4-3.5.Before presenting results on the performance, it is necessary to deal with certain problems that arisein centering and Studentizing statistics based on the block bootstrap. These issues are discussed inSections 3.1.3.1 Centering and Studentizing with the Block BootstrapTwo problems are treated in this section. The first is the construction of a block bootstrapversion of the centered statistic n θ ( mn ) θ ( µ ) that does not have excessive bias.Thisproblem is discussed in Section 3.1.1. The second problem is Studentization of the resulting blockbootstrap version of n1/ 2 n . This is topic of Section 3.2.2. We consider only non-stochastic blocklengths in this section. The stationary bootstrap is discussed in Section 3.3.3.1.1 CenteringThe problem of centering and its solution can be seen most simply by assuming that X i isa scalar and θ is the identity function. Thus, n mn µ and E n 0 . An obvious blockbootstrap version of n is ˆ n mˆ n mn , where m̂n n 1 i 1 Xˆ i ,nand { Xˆ i } is the blockbootstrap sample using either non-overlapping or overlapping blocks.9Let Ê denote the
expectation operator with respect to the probability measure induced by block bootstrap sampling.If the blocks are non-overlapping, then Eˆ mˆ n mn , so Eˆ ˆ n 0 .With overlapping blocks,however, it can be shown thatEˆ mˆ n mn [A( n A 1)] 1[A(A 1) mn τ1 τ 2 ] ,where A is the block length τ1 A 1 j 1(A j) X j ,τ 2 j n A 2 [ j (n A 1)] X j , and it isnassumed for simplicity that n is an integer multiple of A (Hall et al. 1995). Thus, Eˆ m̂n mn withoverlapping blocks. The resulting bias decreases the rate of convergence of the estimation errors ofthe block bootstrap with overlapping blocks.This problem can be solved by centering theoverlapping block bootstrap estimator at Eˆ mˆ n instead of at mn . The resulting bootstrap version of n is ˆ n mˆ n Eˆ mˆ n .θ (mˆ n ) θ ( Eˆ mˆ n ) .More generally, the block bootstrap version of θ (mn ) θ ( µ ) isThis centering can also be used with non-overlapping blocks becauseEˆ mˆ n mn with non-overlapping blocks.3.1.2 StudentizationThis section addresses the problem of Studentizing n1/ 2 ˆ n n1/ 2 [θ (mˆ n ) θ ( Eˆ mˆ n )] . Thesource of the problem is that blocking distorts the dependence structure of the DGP. As a result, themost obvious methods for Studentizing the bootstrap version n1/ 2 n create excessively largeestimation errors. Various forms of this problem have been discussed by Lahiri (1992), Davisonand Hall (1993), and Hall and Horowitz (1996). The discussion in this section is based on Hall andHorowitz (1996).To illustrate the essential issues with a minimum of complexity, assume that the blocks arenon-overlapping, θ is the identity function, and { X i } is a sequence of uncorrelated (though notnecessarily independent) scalar random variables. Let V denote the variance operator relative tothe process that generates { X i } .V (n1/ 2 n ) E ( X1 µ ) 2 .sn2 n 1TheThen n1/ 2 n n1/ 2 (mn µ ) , n1/ 2 ˆ n n1/ 2 ( mˆ n mn ) andnaturalchoiceforsn2isthesamplevariance, i 1 ( X i mn )2 , in which case sn2 Var (n1/ 2 n ) O p (n 1/ 2 ) .nNow consider Studentization of n1/ 2 ˆ n . Let A and B , respectively, denote the blocklength and number of blocks, and assume that BA n . Let Vˆ denote the variance operator relative10
to the block bootstrap DGP. An obvious bootstrap analog of sn2 is sˆn2 n 1 i 1( Xˆ i mˆ n )2 ,nwhich leads to the Studentized statistic T n n1/ 2 ˆ n / sˆn . However, Vˆ (n1/ 2 ˆ n ) s n2 , wheres n2 n 1BAA ( X bA i mn )( X bA j mn )b 0 i 1j 1(Hall and Horowitz 1996). Moroever, sˆn2 s n2 O[(A / n)1/ 2 ] almost surely. The consequences ofthis relatively large error in the estimator of the variance of n1/ 2 ˆ n can be seen by carrying outEdgeworth expansions of the one-sided distribution functionsP (n1/ 2 n / sn z )andPˆ (n1/ 2 ˆ n / sˆn z ) . These have the formsP (n1/ 2 n / sn z ) Φ ( z ) n 1/ 2 g1 ( z ,κ )φ ( z ) O (n 1 )andPˆ (n1/ 2 ˆ n / sˆn z ) Φ ( z ) n 1/ 2 g1 ( z ,κˆ )φ ( z ) zφ ( z )(τ n 1) O (A1/ 2 / n 1/ n)almost surely, where τ n sn / s n and g1 , κ , and κˆ are as defined in Section 2.3 (Hall andHorowitz 1996). Therefore, Pˆ (n1/ 2 ˆ n / sˆn z ) P (n1/ 2 n / sn z ) O[(A / n)1/ 2 ] By contrast, theerror made by first-order asymptotic approximations is O(n 1/ 2 ) . Thus, the block bootstrap doesnot provide asymptotic refinements and, in fact, is less accurate than first-order approximationswhen n1/ 2 ˆ n is Studentized with sˆn .This problem can be mitigated by Studentizing n1/ 2 ˆ n with s n (Lahiri 1996a) or theestimator n 1B 1 b 0 i 1 j 1 ( X bA i mˆ n )( X bA j mˆ n )AA(Götze and Künsch 1996). The errorin the block bootstrap estimator of a one-sided distribution function is then o(n 1/ 2 ) almost surely(Lahiri 1996a, Götze and Künsch 1996).However, the distributions of the symmetricalprobabilities P ( n1/ 2 n / sn z ) and Pˆ ( n1/ 2 ˆ n / s n z ) differ by O(n 1 ) , so the blockbootstrap does not provide asymptotic refinements for symmetrical distributions, confidenceintervals and tests.Refinements for both one-sided and symmetrical distribution functions, confidenceintervals, and tests can be obtained by replacing T n with the “corrected” statistic Tˆn τ nT n (Halland Horowitz 1996, Andrews 2002). In the remainder of this paper, Tˆn will be called a “corrected”11
bootstrap test statistic and τ n will be called a correction factor. The estimation errors resultingfrom the use of corrected statistics are discussed in Section 3.3.The use of a correction factor can be generalized to the case in which θ is not the identityfunction and X i is a vector. Suppose that cov( X i , X j ) 0 whenever i j M for someM . (This assumption is weaker than M-dependence, which requires X i and X j to beindependent when i j M .) Then sn2 θ (mn )′Σ n θ (mn ) , where(3.1)Σ n n 1 ( X i mn )( X i mn )′ i 1 n H ( X i , X i j , mn ) , j 1M and H ( X i , X i j , mn ) ( X i mn )( X i j mn )′ ( X i mn )′( X i j mn ) . The bootstrap version ofsn2 is sˆn2 θ (mˆ n )′Σˆ n θ (mˆ n ) , whereΣˆ n n 1 ( Xˆ i mˆ n )( Xˆ i mˆ n )′ i 1 n M j 1 H ( Xˆ i , Xˆ i j , mˆ n ) .DefineΣ n n 1BAA ( X bA i mn )( X bA j mn )′ ,b 0 i 1j 1s n2 θ (mn )′Σ n θ ( mn ) , and τ n sn / s . Then with non-overlapping blocks, the corrected blockbootstrap version of n1/ 2 (θ n θ ) / sn is τ n n1/ 2 [θ (mˆ n ) θ ( mˆ n )]/ sˆn (Hall and Horowitz 1996).Andrews (2002) gives the overlapping-blocks version of the statistic.The foregoing discussion assumes that cov( X i , X j ) 0 when i j M for someM . When this assumption is not made, Σ n must be replaced by a kernel-type estimator of thecovariance matrix of mn µ . See, for example, (e.g., Newey and West 1987, 1994; Andrews1991, Andrews and Monahan 1992, Götze and Künsch 1996). In contrast to the covariance-matrixestimator (3.1), kernel covariance matrix estimators are not functions of sample moments. Thiscomplicates the analysis of rates of convergence of estimation errors. As was discussed in Section2.3, this analysis is based on Edgeworth expansions of the distributions of the relevant statistics.The most general results on the validity of such expansions assumes that the statistic of interest is afunction of sample moments (Götze and Hipp 1983, 1994; Lahiri 1996b). Consequently, as will bediscussed in Section 3.2, the properties of the block bootstrap are less well understood whenStudentization is with a kernel covariance matrix estimator than when Studentization is with afunction of sample moments.12
3.2 The Accuracy of Block Boo
BOOTSTRAP METHODS FOR TIME SERIES 1. Introduction The bootstrap is a method for estimating the distribution of an estimator or test statistic by resampling one’s data or a model estimated from the data. Under conditions that hold in a wide variety of applications, the bootstrap provides approximations to distributions of statistics,
Bruksanvisning för bilstereo . Bruksanvisning for bilstereo . Instrukcja obsługi samochodowego odtwarzacza stereo . Operating Instructions for Car Stereo . 610-104 . SV . Bruksanvisning i original
The bootstrap distribution of a statistic collects its values from the many resamples. The bootstrap distribution gives information about the sampling distribution. bootstrap distribution 16.2 Bootstrap distribution of mean time to start a business. In Exam-ple 16.1, we want to estimate the population mean time to start a business, m,
the bootstrap, although simulation is an essential feature of most implementations of bootstrap methods. 2 PREHISTORY OF THE BOOTSTRAP 2.1 INTERPRETATION OF 19TH CENTURY CONTRIBUTIONS In view of the definition above, one could fairly argue that the calculation and applica-tion of bootstrap estimators has been with us for centuries.
Bootstrap Cautions These methods for creating a confidence interval only work if the bootstrap distribution is smooth and symmetric ALWAYS look at a plot of the bootstrap distribution! If the bootstrap distribution is highly skewed or looks "spiky" with gaps, you will need to go beyond intro stat to create a confidence interval
validation methods, split-sample methods and the .632 bootstrap for high dimensional genomic studies. In this paper, we compare a number of existing bootstrap methods, the out-of-bag estimation and a bootstrap cross validation method (Fu, Carroll and Wang [9]) for estimating prediction errors when the number of features greatly exceeds the number
know how to create bootstrap weights in Stata and R know how to choose parameters of the bootstrap. Survey bootstrap Stas Kolenikov Bootstrap for i.i.d. data Variance estimation for complex surveys Survey bootstraps Software im-plementation Conclusions References Outline
Chapter 1: Getting started with bootstrap-modal 2 Remarks 2 Examples 2 Installation or Setup 2 Simple Message with in Bootstrap Modal 2 Chapter 2: Examples to Show bootstrap model with different attributes and Controls 4 Introduction 4 Remarks 4 Examples 4 Bootstrap Dialog with Title and Message Only 4 Manipulating Dialog Title 4
The Asset Management Framework table below (Figure 1) encompasses these key documents and illustrates the local and national influences and dependencies that are in place to deliver these services. 3.2 As well as linking in with the City Council’s own vision and objectives, the framework shows the link with the wider objectives of Greater Manchester Combined Authority (GMCA) via its .
