Jonathan D.Cryer Kung-Sik Chan
Springer Texts in StatisticsJonathan D.CryerKung-Sik ChanTime Series AnalysisWith Applications in RSecond Edition
Statistics Texts in StatisticsSeries Editors:G. CasellaS. FienbergI. Olkin
Springer Texts in StatisticsAthreya/Lahiri: Measure Theory and Probability TheoryBilodeau/Brenner: Theory of Multivariate StatisticsBrockwell/Davis: An Introduction to Time Series and ForecastingCarmona: Statistical Analysis of Financial Data in S-PLUSChow/Teicher: Probability Theory: Independence, Interchangeability, Martingales, 3rd ed.Christensen: Advanced Linear Modeling: Multivariate, Time Series, and Spatial Data;Nonparametric Regression and Response Surface Maximization, 2nd ed.Christensen: Log-Linear Models and Logistic Regression, 2nd ed.Christensen: Plane Answers to Complex Questions: The Theory of Linear Models, 2nd ed.Cryer/Chan: Time Series Analysis, Second EditionDavis: Statistical Methods for the Analysis of Repeated MeasurementsDean/Voss: Design and Analysis of ExperimentsDekking/Kraaikamp/Lopuhaä/Meester: A Modern Introduction to Probability and StatisticsDurrett: Essential of Stochastic ProcessesEdwards: Introduction to Graphical Modeling, 2nd ed.Everitt: An R and S-PLUS Companion to Multivariate AnalysisGentle: Matrix Algebra: Theory, Computations, and Applications in StatisticsGhosh/Delampady/Samanta: An Introduction to Bayesian AnalysisGut: Probability: A Graduate CourseHeiberger/Holland: Statistical Analysis and Data Display; An Intermediate Course with Examplesin S-PLUS, R, and SASJobson: Applied Multivariate Data Analysis, Volume I: Regression and Experimental DesignJobson: Applied Multivariate Data Analysis, Volume II: Categorical and Multivariate MethodsKarr: ProbabilityKulkarni: Modeling, Analysis, Design, and Control of Stochastic SystemsLange: Applied ProbabilityLange: OptimizationLehmann: Elements of Large Sample TheoryLehmann/Romano: Testing Statistical Hypotheses, 3rd ed.Lehmann/Casella: Theory of Point Estimation, 2nd ed.Longford: Studying Human Popluations: An Advanced Course in StatisticsMarin/Robert: Bayesian Core: A Practical Approach to Computational Bayesian StatisticsNolan/Speed: Stat Labs: Mathematical Statistics Through ApplicationsPitman: ProbabilityRawlings/Pantula/Dickey: Applied Regression AnalysisRobert: The Bayesian Choice: From Decision-Theoretic Foundations to ComputationalImplementation, 2nd ed.Robert/Casella: Monte Carlo Statistical Methods, 2nd ed.Rose/Smith: Mathematical Statistics with MathematicaRuppert: Statistics and Finance: An IntroductionSen/Srivastava: Regression Analysis: Theory, Methods, and Applications.Shao: Mathematical Statistics, 2nd ed.Shorack: Probability for StatisticiansShumway/Stoffer: Time Series Analysis and Its Applications, 2nd ed.Simonoff: Analyzing Categorical DataTerrell: Mathematical Statistics: A Unified IntroductionTimm: Applied Multivariate AnalysisToutenberg: Statistical Analysis of Designed Experiments, 2nd ed.Wasserman: All of Nonparametric StatisticsWasserman: All of Statistics: A Concise Course in Statistical InferenceWeiss: Modeling Longitudinal DataWhittle: Probability via Expectation, 4th ed.
Jonathan D. Cryer Kung-Sik ChanTime Series AnalysisWith Applications in RSecond Edition
Jonathan D. CryerDepartment of Statistics & Actuarial ScienceUniversity of IowaIowa City, Iowa 52242USAjon-cryer@uiowa.eduKung-Sik ChanDepartment of Statistics & Actuarial ScienceUniversity of IowaIowa City, Iowa 52242USAkung-sik-chan@uiowa.eduSeries Editors:George CasellaDepartment of StatisticsUniversity of FloridaGainesville, FL 32611-8545USAStephen FienbergDepartment of StatisticsCarnegie Mellon UniversityPittsburgh, PA 15213-3890USAISBN: 978-0-387-75958-6e-ISBN: 978-0-387-75959-3Ingram OkinDepartment of StatisticsStanford UniversityStanford, CA 94305USALibrary of Congress Control Number: 2008923058 2008 Springer Science Business Media, LLCAll rights reserved. This work may not be translated or copied in whole or in part without the written permissionof the publisher (Springer Science Business Media, LLC, 233 Spring Street, New York, NY 10013, USA),except for brief excerpts in connection with reviews or scholarly analysis. Use in connection with anyform of information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilarmethodology now known or hereafter developed is forbidden.The use in this publication of trade names, trademarks, service marks, and similar terms, even if they are notidentified as such, is not to be taken as an expression of opinion as to whether or not they are subject toproprietary rights.Printed on acid-free paper.9 8 7 6 5 4 3 2 (Corrected at second printing, 2008)springer.com
To our families
PREFACEThe theory and practice of time series analysis have developed rapidly since the appearance in 1970 of the seminal work of George E. P. Box and Gwilym M. Jenkins, TimeSeries Analysis: Forecasting and Control, now available in its third edition (1994) withco-author Gregory C. Reinsel. Many books on time series have appeared since then, butsome of them give too little practical application, while others give too little theoreticalbackground. This book attempts to present both application and theory at a level accessible to a wide variety of students and practitioners. Our approach is to mix applicationand theory throughout the book as they are naturally needed.The book was developed for a one-semester course usually attended by students instatistics, economics, business, engineering, and quantitative social sciences. Basicapplied statistics through multiple linear regression is assumed. Calculus is assumedonly to the extent of minimizing sums of squares, but a calculus-based introduction tostatistics is necessary for a thorough understanding of some of the theory. However,required facts concerning expectation, variance, covariance, and correlation arereviewed in appendices. Also, conditional expectation properties and minimum meansquare error prediction are developed in appendices. Actual time series data drawn fromvarious disciplines are used throughout the book to illustrate the methodology. The bookcontains additional topics of a more advanced nature that can be selected for inclusion ina course if the instructor so chooses.All of the plots and numerical output displayed in the book have been producedwith the R software, which is available from the R Project for Statistical Computing atwww.r-project.org. Some of the numerical output has been edited for additional clarityor for simplicity. R is available as free software under the terms of the Free SoftwareFoundation's GNU General Public License in source code form. It runs on a wide variety of UNIX platforms and similar systems, Windows, and MacOS.R is a language and environment for statistical computing and graphics, provides awide variety of statistical (e.g., time-series analysis, linear and nonlinear modeling, classical statistical tests) and graphical techniques, and is highly extensible. The extensiveappendix An Introduction to R, provides an introduction to the R software speciallydesigned to go with this book. One of the authors (KSC) has produced a large number ofnew or enhanced R functions specifically tailored to the methods described in this book.They are listed on page 468 and are available in the package named TSA on the RProject’s Website at www.r-project.org. We have also constructed R command scriptfiles for each chapter. These are available for download at www.stat.uiowa.edu/ kchan/TSA.htm. We also show the required R code beneath nearly every table andgraphical display in the book. The datasets required for the exercises are named in eachexercise by an appropriate filename; for example, larain for the Los Angeles rainfalldata. However, if you are using the TSA package, the datasets are part of the packageand may be accessed through the R command data(larain), for example.All of the datasets are also available at the textbook website as ASCII files withvariable names in the first row. We believe that many of the plots and calculationsvii
viiidescribed in the book could also be obtained with other software, such as SAS , Splus ,Statgraphics , SCA , EViews , RATS , Ox , and others.This book is a second edition of the book Time Series Analysis by Jonathan Cryer,published in 1986 by PWS-Kent Publishing (Duxbury Press). This new edition containsnearly all of the well-received original in addition to considerable new material, numerous new datasets, and new exercises. Some of the new topics that are integrated with theoriginal include unit root tests, extended autocorrelation functions, subset ARIMA models, and bootstrapping. Completely new chapters cover the topics of time series regression models, time series models of heteroscedasticity, spectral analysis, and thresholdmodels. Although the level of difficulty in these new chapters is somewhat higher thanin the more basic material, we believe that the discussion is presented in a way that willmake the material accessible and quite useful to a broad audience of users. Chapter 15,Threshold Models, is placed last since it is the only chapter that deals with nonlineartime series models. It could be covered earlier, say after Chapter 12. Also, Chapters 13and 14 on spectral analysis could be covered after Chapter 10.We would like to thank John Kimmel, Executive Editor, Statistics, at Springer, forhis continuing interest and guidance during the long preparation of the manuscript. Professor Howell Tong of the London School of Economics, Professor Henghsiu Tsai ofAcademica Sinica, Taipei, Professor Noelle Samia of Northwestern University, Professor W. K. Li and Professor Kai W. Ng, both of the University of Hong Kong, and Professor Nils Christian Stenseth of the University of Oslo kindly read parts of the manuscript,and Professor Jun Yan used a preliminary version of the text for a class at the Universityof Iowa. Their constructive comments are greatly appreciated. We would like to thankSamuel Hao who helped with the exercise solutions and read the appendix: An Introduction to R. We would also like to thank several anonymous reviewers who read the manuscript at various stages. Their reviews led to a much improved book. Finally, one of theauthors (JDC) would like to thank Dan, Marian, and Gene for providing such a greatplace, Casa de Artes, Club Santiago, Mexico, for working on the first draft of much ofthis new edition.Iowa City, IowaJanuary 2008Jonathan D. CryerKung-Sik Chan
CONTENTSCHAPTER 1 INTRODUCTION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .11.1 Examples of Time Series . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.2 A Model-Building Strategy . . . . . . . . . . . . . . . . . . . . . . . . . . . 81.3 Time Series Plots in History . . . . . . . . . . . . . . . . . . . . . . . . . . 81.4 An Overview of the Book . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10CHAPTER 2 FUNDAMENTAL CONCEPTS . . . . . . . . . . . . . . . . . .112.1 Time Series and Stochastic Processes . . . . . . . . . . . . . . . .2.2 Means, Variances, and Covariances . . . . . . . . . . . . . . . . . .2.3 Stationarity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .2.4 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Appendix A: Expectation, Variance, Covariance, and Correlation .111116191924CHAPTER 3 TRENDS. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 273.1 Deterministic Versus Stochastic Trends . . . . . . . . . . . . . . . .3.2 Estimation of a Constant Mean . . . . . . . . . . . . . . . . . . . . . .3.3 Regression Methods. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .3.4 Reliability and Efficiency of Regression Estimates. . . . . . . .3.5 Interpreting Regression Output . . . . . . . . . . . . . . . . . . . . . .3.6 Residual Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .3.7 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .2728303640425050CHAPTER 4 MODELS FOR STATIONARY TIME SERIES . . . . .554.1 General Linear Processes . . . . . . . . . . . . . . . . . . . . . . . . . .4.2 Moving Average Processes . . . . . . . . . . . . . . . . . . . . . . . . .4.3 Autoregressive Processes . . . . . . . . . . . . . . . . . . . . . . . . . .4.4 The Mixed Autoregressive Moving Average Model. . . . . . . .4.5 Invertibility. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .4.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Appendix B: The Stationarity Region for an AR(2) Process . . . . .Appendix C: The Autocorrelation Function for ARMA(p,q). . . . . . .555766777980818485ix
xContentsCHAPTER 5 MODELS FOR NONSTATIONARY TIME SERIES.875.1 Stationarity Through Differencing . . . . . . . . . . . . . . . . . . . . .885.2 ARIMA Models. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .925.3 Constant Terms in ARIMA Models. . . . . . . . . . . . . . . . . . . . .975.4 Other Transformations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .985.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .102Exercises. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .103Appendix D: The Backshift Operator. . . . . . . . . . . . . . . . . . . . . . .106CHAPTER 6 MODEL SPECIFICATION . . . . . . . . . . . . . . . . . . . . .1096.1 Properties of the Sample Autocorrelation Function . . . . . . .1096.2 The Partial and Extended Autocorrelation Functions . . . . .1126.3 Specification of Some Simulated Time Series. . . . . . . . . . .1176.4 Nonstationarity. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1256.5 Other Specification Methods . . . . . . . . . . . . . . . . . . . . . . . .1306.6 Specification of Some Actual Time Series. . . . . . . . . . . . . .1336.7 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .141Exercises. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .141CHAPTER 7 PARAMETER ESTIMATION . . . . . . . . . . . . . . . . . . .1497.1 The Method of Moments . . . . . . . . . . . . . . . . . . . . . . . . . . .1497.2 Least Squares Estimation . . . . . . . . . . . . . . . . . . . . . . . . . .1547.3 Maximum Likelihood and Unconditional Least Squares . . .1587.4 Properties of the Estimates . . . . . . . . . . . . . . . . . . . . . . . . .1607.5 Illustrations of Parameter Estimation . . . . . . . . . . . . . . . . . .1637.6 Bootstrapping ARIMA Models . . . . . . . . . . . . . . . . . . . . . . .1677.7 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .170Exercises. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .170CHAPTER 8 MODEL DIAGNOSTICS . . . . . . . . . . . . . . . . . . . . . .1758.1 Residual Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1758.2 Overfitting and Parameter Redundancy. . . . . . . . . . . . . . . .1858.3 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .188Exercises. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .188
ContentsxiCHAPTER 9 FORECASTING . . . . . . . . . . . . . . . . . . . . . . . . . . . . .9.19.29.39.49.59.69.7191Minimum Mean Square Error Forecasting . . . . . . . . . . . . .Deterministic Trends . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .ARIMA Forecasting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Prediction Limits . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Forecasting Illustrations . . . . . . . . . . . . . . . . . . . . . . . . . . .Updating ARIMA Forecasts . . . . . . . . . . . . . . . . . . . . . . . .Forecast Weights and Exponentially WeightedMoving Averages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .9.8 Forecasting Transformed Series. . . . . . . . . . . . . . . . . . . . .9.9 Summary of Forecasting with Certain ARIMA Models . . . .9.10 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Appendix E: Conditional Expectation. . . . . . . . . . . . . . . . . . . . . .Appendix F: Minimum Mean Square Error Prediction . . . . . . . . .Appendix G: The Truncated Linear Process . . . . . . . . . . . . . . . .Appendix H: State Space Models . . . . . . . . . . . . . . . . . . . . . . . .191191193203204207CHAPTER 10 SEASONAL MODELS . . . . . . . . . . . . . . . . . . . . . .22710.1 Seasonal ARIMA Models . . . . . . . . . . . . . . . . . . . . . . . . . .10.2 Multiplicative Seasonal ARMA Models . . . . . . . . . . . . . . . .10.3 Nonstationary Seasonal ARIMA Models . . . . . . . . . . . . . .10.4 Model Specification, Fitting, and Checking. . . . . . . . . . . . .10.5 Forecasting Seasonal Models . . . . . . . . . . . . . . . . . . . . . .10.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .228230233234241246246CHAPTER 11 TIME SERIES REGRESSION MODELS207209211213213218218221222. . . . . . 24911.1 Intervention Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . .11.2 Outliers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .11.3 Spurious Correlation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .11.4 Prewhitening and Stochastic Regression . . . . . . . . . . . . . .11.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .249257260265273274
xiiContentsCHAPTER 12 TIME SERIES MODELS OFHETEROSCEDASTICITY . . . . . . . . . . . . . . . . . . . . .27712.112.212.312.412.512.6Some Common Features of Financial Time Series . . . . . . .278The ARCH(1) Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .285GARCH Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .289Maximum Likelihood Estimation . . . . . . . . . . . . . . . . . . . . .298Model Diagnostics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .301Conditions for the Nonnegativity of theConditional Variances . . . . . . . . . . . . . . . . . . . . . . . . . . . . .30712.7 Some Extensions of the GARCH Model . . . . . . . . . . . . . . .31012.8 Another Example: The Daily USD/HKD Exchange Rates . .31112.9 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .315Exercises. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .316Appendix I: Formulas for the Generalized Portmanteau Tests . . .318CHAPTER 13 INTRODUCTION TO SPECTRAL ANALYSIS. . . .31913.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .31913.2 The Periodogram . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .32213.3 The Spectral Representation and Spectral Distribution . . . .32713.4 The Spectral Density . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .33013.5 Spectral Densities for ARMA Processes . . . . . . . . . . . . . . .33213.6 Sampling Properties of the Sample Spectral Density . . . . .34013.7 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .346Exercises. . . . . .
Everitt: An R and S-PLUS Companion to Multivariate Analysis Gentle: Matrix Algebra: Theory, Computations, and Applications in Statistics Ghosh/Delampady/Samanta: An Introduction to Bayesian Analysis Gut: Probability: A Graduate Course in S-PLUS, R, and SAS Jobson: Applied Multivariate Data Analys
Kung ikaw ay nasa direktang panganib, tumawag ng pulis sa 911 (sa loob ng Estados Unidos). Sabihin sa kanila kung ano ang emergency, kung nasaan ka, at ang telephone number na kung saan tumatawag. Humingi ng interpreter kung hindi marunong mag-Ingles. Pagdating ng pulis, ipakita sa kanila ang pamphlet na ito at sabihin sa kanila ang pang-
KUNG FU PANDA Written By Jonathan Aibel & Glenn Berger FINAL DRAFT June 3, 2008. EXT. VALLEY -- DAY A MYSTERIOUS WARRIOR treks across the rugged landscape. NARRATOR (V.O.) Legend tells of a legendary warrior whose Kung Fu skills were the stuff of legend. The warrior, his identity hidden beneath his flowing robe and
Kung fu isn’t supposed to work for lone, skinny guys against gangs of drug dealers in bad neighborhoods – you need a posse and automatic weapons for that. Anyway, this student quit his kung fu training shortly after that episode, perhaps permanently disheartened to find that his kung fu didn’t work.
Other Kung Fu Panda films: Kung Fu Panda Holiday (TV short: 2010), Kung Fu Panda 2 (2011), Kung Fu Panda 3 (2016) Starter Level Popcorn ELT Readers Starter level is for children who have just started learning English at primary school. Each Starter level title is written to a 150 headword list. There are no past tenses at this level.
from the local Kung-Fu school. The school teaches Kung-Fu but due to the large spiritual component and nature, a large amount of Tai Chi is also included in the training. Inclusion criteria were that the participants had practiced Kung-Fu and Tai Chi for a minimum of 1 year and successfully passed the medical screening for participation.
KUNG FU PANDA Written By Jonathan Aibel & Glenn Berger FINAL DRAFT June 3, 2008 1. EXT. VALLEY -- DAY A MYSTERIOUS WARRIOR treks across the rugged landscape. NARRATOR (V.O.) Legend tells of a legendary warrior whose Kung Fu skills were the stuff of legend. The warrior, his identity hidden beneath his flowing robe and
wrath of King Saul. As David hid in a field, Jonathan went to the festival with his father, the king, and watched for signs of Saul's hatred toward David. Jonathan at first couldn't believe that his father wanted to harm David. When Jonathan discovered how fervently Saul wished for David's death, Jonathan was so troubled that he couldn .
The themes of pilgrimage and welcome are central to The Canterbury Journey. A lasting part of its legacy will be the new free-to-enter Welcome Centre with dedicated community and exhibition spaces and viewing gallery. The journey to our new centre is underway, to open in 2019. A New Welcome In 2017, the face of the Cathedral has changed .
