Time Series Analysis - Auckland
Time Series AnalysisLecture Notes for 475.726Ross IhakaStatistics DepartmentUniversity of AucklandApril 14, 2005
ii
Contents1 Introduction1.1 Time Series . . . . . . . . . . . . . . . . . . . .1.2 Stationarity and Non-Stationarity . . . . . . .1.3 Some Examples . . . . . . . . . . . . . . . . . .1.3.1 Annual Auckland Rainfall . . . . . . . .1.3.2 Nile River Flow . . . . . . . . . . . . . .1.3.3 Yield on British Government Securities1.3.4 Ground Displacement in an Earthquake1.3.5 United States Housing Starts . . . . . .1.3.6 Iowa City Bus Ridership . . . . . . . . .11122223332 Vector Space Theory2.1 Vectors In Two Dimensions . . . . . . . .2.1.1 Scalar Multiplication and Addition2.1.2 Norms and Inner Products . . . .2.2 General Vector Spaces . . . . . . . . . . .2.2.1 Vector Spaces and Inner Products2.2.2 Some Examples . . . . . . . . . . .2.3 Hilbert Spaces . . . . . . . . . . . . . . .2.3.1 Subspaces . . . . . . . . . . . . . .2.3.2 Projections . . . . . . . . . . . . .2.4 Hilbert Spaces and Prediction . . . . . . .2.4.1 Linear Prediction . . . . . . . . . .2.4.2 General Prediction . . . . . . . . .777899111213131414153 Time Series Theory3.1 Time Series . . . . . . . . . . . . . . . . .3.2 Hilbert Spaces and Stationary Time Series3.3 The Lag and Differencing Operators . . .3.4 Linear Processes . . . . . . . . . . . . . .3.5 Autoregressive Series . . . . . . . . . . . .3.5.1 The AR(1) Series . . . . . . . . . .3.5.2 The AR(2) Series . . . . . . . . . .3.5.3 Computations . . . . . . . . . . . .3.6 Moving Average Series . . . . . . . . . . .3.6.1 The MA(1) Series . . . . . . . . .3.6.2 Invertibility . . . . . . . . . . . . .3.6.3 Computation . . . . . . . . . . . .17171819202121232629292930iii
ivContents3.7.30313232343436374 Identifying Time Series Models4.1 ACF Estimation . . . . . . . . . . . . . . . . . . . . .4.2 PACF Estimation . . . . . . . . . . . . . . . . . . . . .4.3 System Identification . . . . . . . . . . . . . . . . . . .4.4 Model Generalisation . . . . . . . . . . . . . . . . . . .4.4.1 Non-Zero Means . . . . . . . . . . . . . . . . .4.4.2 Deterministic Trends . . . . . . . . . . . . . . .4.4.3 Models With Non-stationary AR Components .4.4.4 The Effect of Differencing . . . . . . . . . . . .4.5 ARIMA Models . . . . . . . . . . . . . . . . . . . . . .393942434545474748495 Fitting and Forecasting5.1 Model Fitting . . . . . . . . . . . . . . . . . . . . . . . . . .5.1.1 Computations . . . . . . . . . . . . . . . . . . . . . .5.2 Assessing Quality of Fit . . . . . . . . . . . . . . . . . . . .5.3 Residual Correlations . . . . . . . . . . . . . . . . . . . . .5.4 Forecasting . . . . . . . . . . . . . . . . . . . . . . . . . . .5.4.1 Computation . . . . . . . . . . . . . . . . . . . . . .5.5 Seasonal Models . . . . . . . . . . . . . . . . . . . . . . . .5.5.1 Purely Seasonal Models . . . . . . . . . . . . . . . .5.5.2 Models with Short-Term and Seasonal Components5.5.3 A More Complex Example . . . . . . . . . . . . . .51515154585959616163666 Frequency Domain Analysis6.1 Some Background . . . . . . . . . . . . . . . . .6.1.1 Complex Exponentials, Sines and Cosines6.1.2 Properties of Cosinusoids . . . . . . . . .6.1.3 Frequency and Angular Frequency . . . .6.1.4 Invariance and Complex Exponentials . .6.2 Filters and Filtering . . . . . . . . . . . . . . . .6.2.1 Filters . . . . . . . . . . . . . . . . . . . .6.2.2 Transfer Functions . . . . . . . . . . . . .6.2.3 Filtering Sines and Cosines . . . . . . . .6.2.4 Filtering General Series . . . . . . . . . .6.2.5 Computing Transfer Functions . . . . . .6.2.6 Sequential Filtering . . . . . . . . . . . .6.3 Spectral Theory . . . . . . . . . . . . . . . . . . .6.3.1 The Power Spectrum . . . . . . . . . . . .6.3.2 The Cramér Representation . . . . . . . .6.3.3 Using The Cramér Representation . . . .6.3.4 Power Spectrum Examples . . . . . . . sive Moving Average Series3.7.1 The ARMA(1,1) Series . . . .3.7.2 The ARMA(p,q) Model . . . .3.7.3 Computation . . . . . . . . . .3.7.4 Common Factors . . . . . . . .The Partial Autocorrelation Function3.8.1 Computing the PACF . . . . .3.8.2 Computation . . . . . . . . . .
Contents6.46.56.6vStatistical Inference . . . . . . . . . . . . . . . .6.4.1 Some Distribution Theory . . . . . . . . .6.4.2 The Periodogram and its Distribution . .6.4.3 An Example – Sunspot Numbers . . . . .6.4.4 Estimating The Power Spectrum . . . . .6.4.5 Tapering and Prewhitening . . . . . . . .6.4.6 Cross Spectral Analysis . . . . . . . . . .Computation . . . . . . . . . . . . . . . . . . . .6.5.1 A Simple Spectral Analysis Package for R6.5.2 Power Spectrum Estimation . . . . . . . .6.5.3 Cross-Spectral Analysis . . . . . . . . . .Examples . . . . . . . . . . . . . . . . . . . . . . 88. 88. 90. 91. 92. 95. 96. 99. 99. 99. 100. 100
viContents
Chapter 1Introduction1.1Time SeriesTime series arise as recordings of processes which vary over time. A recordingcan either be a continuous trace or a set of discrete observations. We willconcentrate on the case where observations are made at discrete equally spacedtimes. By appropriate choice of origin and scale we can take the observationtimes to be 1, 2, . . . T and we can denote the observations by Y1 , Y2 , . . . , YT .There are a number of things which are of interest in time series analysis.The most important of these are:Smoothing: The observed Yt are assumed to be the result of “noise” valuesεt additively contaminating a smooth signal ηt .Yt ηt εtWe may wish to recover the values of the underlying ηt .Modelling: We may wish to develop a simple mathematical model whichexplains the observed pattern of Y1 , Y2 , . . . , YT . This model may dependon unknown parameters and these will need to be estimated.Forecasting: On the basis of observations Y1 , Y2 , . . . , YT , we may wish topredict what the value of YT L will be (L 1), and possibly to give anindication of what the uncetainty is in the prediction.Control: We may wish to intervene with the process which is producing theYt values in such a way that the future values are altered to produce afavourable outcome.1.2Stationarity and Non-StationarityA key idea in time series is that of stationarity. Roughly speaking, a timeseries is stationary if its behaviour does not change over time. This means, forexample, that the values always tend to vary about the same level and thattheir variability is constant over time. Stationary series have a rich theory and1
2Chapter 1. Introductiontheir behaviour is well understood. This means that they play a fundamentalrole in the study of time series.Obviously, not all time series that we encouter are stationary. Indeed, nonstationary series tend to be the rule rather than the exception. However, manytime series are related in simple ways to series which are stationary. Two important examples of this are:Trend models : The series we observe is the sum of a determinstic trendseries and a stationary noise series. A simple example is the linear trendmodel:Yt β0 β1 t εt .Another common trend model assumes that the series is the sum of aperiodic “seasonal” effect and stationary noise. There are many othervariations.Integrated models : The time series we observe satisfiesYt 1 Yt εt 1where εt is a stationary series. A particularly important model of this kindis the random walk. In that case, the εt values are independent “shocks”which perturb the current state Yt by an amount εt 1 to produce a newstate Yt 1 .1.31.3.1Some ExamplesAnnual Auckland RainfallFigure 1.1 shows the annual amount of rainfall in Auckland for the years from1949 to 2000. The general pattern of rainfall looks similar throughout the record,so this series could be regarded as being stationary. (There is a hint that rainfallamounts are declining over time, but this type of effect can occur over shortishtime spans for stationary series.)1.3.2Nile River FlowFigure 1.2 shows the flow volume of the Nile at Aswan from 1871 to 1970. Theseare yearly values. The general pattern of this data does not change over timeso it can be regarded as stationary (at least over this time period).1.3.3Yield on British Government SecuritiesFigure 1.3 shows the percentage yield on British Government securities, monthlyover a 21 year period. There is a steady long-term increase in the yields. Overthe period of observation a trend-plus-stationary series model looks like it mightbe appropriate. An integrated stationary series is another possibility.
1.3. Some Examples1.3.43Ground Displacement in an EarthquakeFigure 1.4 shows one component of the horizontal ground motion resulting froman earthquake. The initial motion (a little after 4 seconds) corresponds to thearrival of the p-wave and the large spike just before six seconds corresponds tothe arrival of the s-wave. Later features correspond to the arrival of surfacewaves. This is an example of a transient signal and cannot have techniquesappropriate for stationary series applied to it.1.3.5United States Housing StartsFigure 1.5 shows the monthly number of housing starts in the Unites States (inthousands). Housing starts are a leading economic indicator. This means thatan increase in the number of housing starts indicates that economic growth islikely to follow and a decline in housing starts indicates that a recession may beon the way.1.3.6Iowa City Bus RidershipFigure 1.6 shows the monthly average weekday bus ridership for Iowa City overthe period from September 1971 to December 1982. There is clearly a strongseasonal effect suprimposed on top of a general upward trend.
4Chapter 1. Introduction200Annual Rainfall igure 1.1: Annual Auckland rainfall (in cm) from 1949 to 2000 (fromPaul 01960YearFigure 1.2: Flow volume of the Nile at Aswan from 1871 to 1970 (fromDurbin and Koopman).
1.3. Some Examples5Percent8642050100150200250MonthFigure 1.3: Monthly percentage yield on British Government securitiesover a 21 year period (from Chatfield).500400Displacement3002001000 100024681012SecondsFigure 1.4: Horizontal ground displacement during a small Nevadaearthquake (from Bill Peppin).
6Chapter 1. IntroductionHousing Starts (000s)2001501005019661968197019721974TimeFigure 1.5: Housing starts in the United States (000s) (from S-Plus).Average Weekly TimeFigure 1.6: Average weekday bus ridership, Iowa City (monthly ave)Sep 1971 - Dec 1982.
Chapter 2Vector Space Theory2.1Vectors In Two DimensionsThe theory which underlies time series analysis is quite technical in nature.In spite of this, a good deal of intuition can be developed by approaching thesubject geometrically. The geometric approach is based on the ideas of vectorsand vector spaces.2.1.1Scalar Multiplication and AdditionA good deal can be learnt about the theory of vectors by considering the twodimensional case. You can think of two dimensional vectors as being littlearrows which have a length and a direction. Individual vectors can be stretched(altering their length but not their direction) and pairs of vectors can be addedby placing them head to tail.To get more precise, we’ll suppose that a vector v extends for a distance xin the horizontal direction and a distance y in the vertical direction. This givesus a representation of the vector as a pair of numbers and we can write it asv (x, y).Doubling the length of the vector doubles the x and y values. In a similarway we can scale the length of the vector by any value, and this results in asimilar scaling of the x and y values. This gives us a natural way of definingmultiplication of a vector by a number.cv (cx, cy)A negative value for c produces a reversal of direction as well as change of lengthof magnitude c .Adding two vectors amounts to placing them head to tail and taking the sumto be the arrow which extends from the base of the first to the head of the second.This corresponds to adding the x and y values corresponding to the vectors. Fortwo vectors v 1 (x1 , y1 ) and v 2 (x2 , y2 ) the result is (x1 x2 , y1 y2 ). Thiscorresponds to a natural definition of addition for vectors.v 1 v 2 (x1 x2 , y1 y2 )7
82.1.2Chapter 2. Vector Space TheoryNorms and Inner ProductsWhile coordinates give a complete description of vectors, it is often more usefulto describe them in terms of the lengths of individual vectors and the anglesbetween pairs of vectors. If we denote the length of a vector by kuk, then byPythagoras’ theorem we knowpkuk x2 y 2 .Vector length has a number of simple properties:Positivity: For every vector u, we must have kuk 0, with equality if andonly if u 0.Linearity: For every scalar c and vector u we have kcuk c kuk.The Triangle Inequality: If u and v are vectors, then ku vk 6 kuk kvk.The first and second of these properties are obvious, and the third is simplya statement that the shortest distance between two points is a straight line.The technical mathematical name for the length of a vector is the norm of thevector.Although the length of an individual vector gives some information aboutit, it is also important to consider the angles between pairs of vectors. Supposethat we have vectors v 1 (x1 , y1 ) and v 2 (x2 , y2 ), and that we want to knowthe angle between them. If v 1 subtends an angle θ1 with the x axis and v 2subtends an angle θ2 with the x axis, simple geometry tells us that:cos θi xi,kv i ksin θi yi.kv i kThe angle we are interested in is θ1 θ2 and the cosine of this can be computedusing the formulae:cos(α β) cos α cos β sin α sin βsin(α β) sin α cos β cos α sin β,so thatx1 x2 y1 y2kv 1 kkv 2 kx2 y1 x1 y2.sin(θ1 θ2 ) kv 1 kkv 2 kcos(θ1 θ2 ) Knowledge of the sine and cosine values makes it possible to compute the anglebetween the vectors.There are actually two angles between a pair of vectors; one measured clockwise and one measured counter-clockwise. Often we are interested in the smallerof these two angles. This can be determined from the cosine value (becausecos θ cos θ). The cosine is determined by the lengths of the two vectorsand the quantity x1 x2 y1 y2 . This quantity is called the inner product of thevectors v 1 and v 2 and is denoted by hv 1 , v 2 i. Inner products have the followingbasic properties.
2.2. General Vector Spaces9Positivity: For every vector v, we must have hv, vi 0, with hv, vi 0 onlywhen v 0.Linearity: For all vectors u, v, and w; and scalars α and β; we must havehαu βv, wi αhu, wi βhv, wi.Symmetry: For all vectors u and v, we must have hu, vi hv, ui (or in thecase of complex vector spaces hu, vi hv, ui, where the bar indicates acomplex conjugate).It is clear that the norm can be defined in terms of the inner product bykvk2 hv, vi.It is easy to show that the properties of the norm kvk follow from those of theinner product.Two vectors u and v are said to be orthogonal if hu, vi 0. This meansthat the vectors are “at right angles” to each other.2.2General Vector SpacesThe theory and intuition obtained from studying two-dimensional vectors carries over into more general situations. It is not the particular representationof vectors as pairs of coordinates which is important but rather the ideas ofaddition, scalar multiplication and inner-product which are important.2.2.1Vector Spaces and Inner ProductsThe following concepts provide an abstract generalisation of vectors in two dimensions.1. A vector space is a set of objects, called vectors, which is closed underaddition and scalar multiplication. This means that when vectors arescaled or added, the result is a vector.2. An inner product on a vector space is a function which takes two vectorsu and v and returns a scalar denoted by h u,v i which has the conditionsof positivity, linearity and symmetry described in section 2.1.2. A vectorspace with an associated inner product is called an inner product space.3. The norm associated with anp inner product space is defined in terms ofthe inner product as kuk hu, ui.These ideas can be applied to quite general vector spaces. Although it mayseem strange to apply ideas of direction and length to some of these spaces,thinking in terms of two and three dimensional pictures can be quite useful.Example 2.2.1 The concept of two-dimensional vectors can be generalised bylooking at the set of n-tuples of the form u (u1 , u2 , . . . , un ), where each ui isa real number. With addition and scalar multiplication defined element-wise,
10Chapter 2. Vector Space Theorythe set of all such n-tuples forms a vector space, usually denoted by Rn . Aninner product can be defined on the space byhu, vi nXui vi ,i 1and this produces the normkuk nX! 21u2i.i 1This generalisation of vector ideas to n dimensions provides the basis for agood deal of statistical theory. This is especially true for linear models andmultivariate analysis.We will need a number of fundamental results on vector spaces. These followdirectly from the definition of vector space, inner-product and norm, and do notrely on any special assumptions about the kind of vectors being considered.The Cauchy-Schwarz Inequality. For any two vectors u and v hu, vi 6 kukkvk.(2.1)Proof : For any two vectors u and v and scalars α and β,0 6 kαu βvk2 hαu βv, αu βvi α2 kuk2 2αβhu, vi β 2 kvk2Setting α kvk and β hu, vi/kvk yields0 6 kuk2 kvk2 2hu, vi2 hu, vi2 kuk2 kvk2 hu, vi2 .This can be rewritten as hu, vi 6 kukkvk.The Triangle Inequality. For any two vectors u and v,ku vk 6 kuk kvk.Proof : For any u and vku vk2 hu v, u vi hu, ui hu, vi hv, ui hv, vi6 kuk2 2kukkvk kvk2(Cauchy-Schwarz)2 (kuk kvk)The Angle Between Vectors. The Cauchy-Schwarz inequality means that 1 6hx, yi6 1.kxkkykThis means that just as in the 2-d case we can define the angle betweenvectors u and v byhx, yiθ cos 1.kxkkyk
2.2. General Vector Spaces2.2.211Some ExamplesThe ideas in abstract vector space theory are derived from the concrete ideas ofvectors in 2 and 3 dimensions. The real power of vector space theory is that itapplies to much more general spaces. To show off the generality of the theorywe’ll now look at a couple of examples of less concrete spaces.Example 2.2.2 The set F of all (real-valued) random variables with EX 02and E X is a vector space, because for any choice of X1 and X2 from Fand scalars β1 and β2E[β1 X1 β2 X2 ] 0andE β1 X1 β2 X2 2 .It is possible to define an inner product on this vector space byhX, Y i EXY.This in turn produces the normkXk E X 2 21,which is just the standard deviation of X.The cosine of the angle between random variables is defined asEXYpE X 2 E Y 2,which is recognisable as the correlation between the X and Y .(There is a technical problem with uniqueness in this case. Any randomvariable which has probability 1 of being zero will have hX, Xi 0, whichviolates the requirement that this only happen for X 0. The workaround is toregard random variables as identical if they are the
time series are related in simple ways to series which are stationary. Two im-portant examples of this are: Trend models : The series we observe is the sum of a determinstic trend series and a stationary noise series. A simple example is the linear trend model: Y t β 0 β 1t ε t. Another common trend model assumes that the series is the .
PROPERTY PROFILE REPORT 1 Sample Road, Albany, Auckland, 0632 Prepared on 08 July 2020 PLESAMPLE. 1 Sample Road, Albany, Auckland, 0632 3 1 2 210m2 612m2 Property Details . Email Us: help@corelogic.co.nz 1 Sample Road, Albany, Auckland, 0632 CoreLogic Property Profile Report
bistro bambina 268 ponsonby rd ponsonby 1011 central auckland blue elephant thai restaurant 237 parnell road parnell 1052 central auckland . cibo 91 st georges bay road parnell 1052 central auckland . jervois rd wine bar & kitchen 170 jervois rd herne bay 1011 central auckland
Dec 11, 2014 · Mob: 021 969 4444 Email: George.Hawkins@aucklandcou ncil.govt.nz Ken Penney 146e Great South Road Manurewa, Auckland 2012 Mob: 021 287 2244 Email: Ken.Penney@aucklandcouncil.govt.nz Angela Cunningham –Marino C/- Auckland Council Private Bag 92300 Auckland 1142
proceedings Proceedings Integrating the OpenSky Network into GNSS-R Climate Monitoring Research † Mike Laverick1,* , Delwyn Moller 2, Christopher Ruf 3, Stephen Musko 3, Andrew O'Brien 4, Ryan Linnabary 4, Wayne Thomas 5, Chris Seal 1 and Yvette Wharton 1 1 Centre for eResearch, The University of Auckland, Auckland 1010, New Zealand; c.seal@auckland.ac.nz (C.S.); y.wharton@auckland.ac.nz .
SMB_Dual Port, SMB_Cable assembly, Waterproof Cap RF Connector 1.6/5.6 Series,1.0/2.3 Series, 7/16 Series SMA Series, SMB Series, SMC Series, BT43 Series FME Series, MCX Series, MMCX Series, N Series TNC Series, UHF Series, MINI UHF Series SSMB Series, F Series, SMP Series, Reverse Polarity
LSI Auckland Curriculum and Course Descriptions 2013 English for Academic Purposes at LSI Auckland 30 lessons per week Intermediate to Advanced Levels 1 lesson: 50 minutes 15 students max per class Course length 12 weeks Age 16 The EAP programme is designed for High School or University graduates who need to meet English
Aiken Douglas Ronald Alexander 26 Okona Crescent Ngongotaha . Anderson Dr William Arthur OBE 11 Stanley Street Queenstown . Babington Arnold Oscar 58 Margot Street Epsom Auckland 3 Babington Mrs Florence Lilian 108 Paihia Road Onehunga Auckland 6 Auckland .
3 Royal Forest and Bird Protection Society of NZ, P.O. Box 108055, Symonds Street, Auckland, New Zealand. 4 University of Auckland, School of Biological Sciences, PB 92019, Auckland, New Zealand. Ab
