Time Series Analysis
Description
More details
Other editions
Additional editions
Previous edition
Persons
The late George E. P. Box, PhD, was professor emeritus of statistics at the University of Wisconsin-Madison. He was a Fellow of the American Academy of Arts and Sciences and a recipient of the Samuel S. Wilks Memorial Medal of the American Statistical Association, the Shewhart Medal of the American Society for Quality, and the Guy Medal in Gold of the Royal Statistical Society. Dr. Box was also author of seven Wiley books.
The late Gwilym M. Jenkins, PhD, was professor of systems engineering at Lancaster University in the United Kingdom, where he was also founder and managing director of the International Systems Corporation of Lancaster. A Fellow of the Institute of Mathematical Statistics and the Institute of Statisticians, Dr. Jenkins had a prestigious career in both academia and consulting work that included positions at Imperial College London, Stanford University, Princeton University, and the University of Wisconsin-Madison. He was widely known for his work on time series analysis, most notably his groundbreaking work with Dr. Box on the Box-Jenkins models.
The late Gregory C. Reinsel, PhD, was professor and former chair of the department of Statistics at the University of Wisconsin-Madison. Dr. Reinsel's expertise was focused on time series analysis and its applications in areas as diverse as economics, ecology, engineering, and meteorology. He authored over seventy refereed articles and three books, and was a Fellow of both the American Statistical Association and the Institute of Mathematical Statistics.
Greta M. Ljung, PhD, is a statistical consultant residing in Lexington, MA. She received her doctorate from the University of Wisconsin-Madison where she did her research in time series analysis under the direction of Professor George Box. Dr. Ljung's career includes teaching positions at Boston University and Massachusetts Institute of Technology, and a position as Principal Scientist at AIR Worldwide in Boston. Her many accomplishments include joint work with George Box on a time series goodness of fit test, which is widely applied in econometrics and other fields.
Content
Preface to the Fifth Edition xix
Preface to the Fourth Edition xxiii
Preface to the Third Edition xxv
1 Introduction 1
1.1 Five Important Practical Problems 2
1.2 Stochastic and Deterministic Dynamic Mathematical Models 6
1.3 Basic Ideas in Model Building 14
Appendix A. 1 Use of the R Software 17
Exercises 18
Part One Stochastic Models and Their Forecasting 19
2 Autocorrelation Function and Spectrum of Stationary Processes 21
2.1 Autocorrelation Properties of Stationary Models 21
2.2 Spectral Properties of Stationary Models 34
Appendix A2. 1 Link Between the Sample Spectrum and Autocovariance Function Estimate 43
Exercises 44
3 Linear Stationary Models 47
3.1 General Linear Process 47
3.2 Autoregressive Processes 54
3.3 Moving Average Processes 68
3.4 Mixed Autoregressive--Moving Average Processes 75
Appendix A3. 1 Autocovariances Autocovariance Generating Function, and Stationarity Conditions for a General Linear Process 82
Appendix A3. 2 Recursive Method for Calculating Estimates of Autoregressive Parameters 84
Exercises 86
4 Linear Nonstationary Models 88
4.1 Autoregressive Integrated Moving Average Processes 88
4.2 Three Explicit Forms for the ARIMA Model 97
4.3 Integrated Moving Average Processes 106
Appendix A4. 1 Linear Difference Equations 116
Appendix A4. 2 IMA(0, 1, 1) Process with Deterministic Drift 121
Appendix A4. 3 ARIMA Processes with Added Noise 122
Exercises 126
5 Forecasting 129
5.1 Minimum Mean Square Error Forecasts and Their Properties 129
5.2 Calculating Forecasts and Probability Limits 135
5.3 Forecast Function and Forecast Weights 139
5.4 Examples of Forecast Functions and Their Updating 144
5.5 Use of State-Space Model Formulation for Exact Forecasting 155
5.6 Summary 162
Appendix A5. 1 Correlation Between Forecast Errors 164
Appendix A5. 2 Forecast Weights for any Lead Time 166
Appendix A5. 3 Forecasting in Terms of the General Integrated Form 168
Exercises 174
Part Two STOCHASTIC MODEL BUILDING 177
6 Model Identification 179
6.1 Objectives of Identification 179
6.2 Identification Techniques 180
6.3 Initial Estimates for the Parameters 194
6.4 Model Multiplicity 202
Appendix A6. 1 Expected Behavior of the Estimated Autocorrelation Function for a Nonstationary Process 206
Exercises 207
7 Parameter Estimation 209
7.1 Study of the Likelihood and Sum-of-Squares Functions 209
7.2 Nonlinear Estimation 226
7.3 Some Estimation Results for Specific Models 236
7.4 Likelihood Function Based on the State-Space Model 242
7.5 Estimation Using Bayes' Theorem 245
Appendix A7. 1 Review of Normal Distribution Theory 251
Appendix A7. 2 Review of Linear Least-Squares Theory 256
Appendix A7. 3 Exact Likelihood Function for Moving Average and Mixed Processes 259
Appendix A7. 4 Exact Likelihood Function for an Autoregressive Process 266
Appendix A7. 5 Asymptotic Distribution of Estimators for Autoregressive Models 274
Appendix A7. 6 Examples of the Effect of Parameter Estimation Errors on Variances of Forecast Errors and Probability Limits for Forecasts 277
Appendix A. 7 Special Note on Estimation of Moving Average Parameters 280
Exercises 280
8 Model Diagnostic Checking 284
8.1 Checking the Stochastic Model 284
8.2 Diagnostic Checks Applied to Residuals 287
8.3 Use of Residuals to Modify the Model 301
Exercises 303
9 Analysis of Seasonal Time Series 305
9.1 Parsimonious Models for Seasonal Time Series 305
9.2 Representation of the Airline Data by a Multiplicative (0, 1, 1) × (0, 1, 1) 12 Model 310
9.3 Some Aspects of More General Seasonal ARIMA Models 325
9.4 Structural Component Models and Deterministic Seasonal Components 331
9.5 Regression Models with Time Series Error Terms 339
Appendix A9. 1 Autocovariances for Some Seasonal Models 345
Exercises 349
10 Additional Topics and Extensions 352
10.1 Tests for Unit Roots in ARIMA Models 353
10.2 Conditional Heteroscedastic Models 361
10.3 Nonlinear Time Series Models 377
10.4 Long Memory Time Series Processes 385
Exercises 392
Part Three Transfer Function and Multivariate Model Building 395
11 Transfer Function Models 397
11.1 Linear Transfer Function Models 397
11.2 Discrete Dynamic Models Represented by Difference Equations 404
11.3 Relation Between Discrete and Continuous Models 414
Appendix A11. 1 Continuous Models with Pulsed Inputs 420
Appendix A11. 2 Nonlinear Transfer Functions and Linearization 424
Exercises 426
12 Identification, Fitting, and Checking of Transfer Function Models 428
12.1 Cross-Correlation Function 429
12.2 Identification of Transfer Function Models 435
12.3 Fitting and Checking Transfer Function Models 446
12.4 Some Examples of Fitting and Checking Transfer Function Models 453
12.5 Forecasting with Transfer Function Models Using Leading Indicators 461
12.6 Some Aspects of the Design of Experiments to Estimate Transfer Functions 469
Appendix A12.1 Use of Cross-Spectral Analysis for Transfer Function Model Identification 471
Appendix A12.2 Choice of Input to Provide Optimal Parameter Estimates 473
Exercises 477
13 Intervention Analysis, Outlier Detection, and Missing Values 481
13.1 Intervention Analysis Methods 481
13.2 Outlier Analysis for Time Series 488
13.3 Estimation for ARMA Models with Missing Values 495
Exercises 502
14 Multivariate Time Series Analysis 505
14.1 Stationary Multivariate Time Series 506
14.2 Vector Autoregressive Models 509
14.3 Vector Moving Average Models 524
14.4 Vector Autoregressive--Moving Average Models 527
14.5 Forecasting for Vector Autoregressive--Moving Average Processes 534
14.6 State-Space Form of the VARMA Model 536
14.7 Further Discussion of VARMA Model Specification 539
14.8 Nonstationarity and Cointegration 546
Appendix A14. 1 Spectral Characteristics and Linear Filtering Relations for Stationary Multivariate Processes 552
Exercises 554
Part Four Design of Discrete Control Schemes 559
15 Aspects of Process Control 561
15.1 Process Monitoring and Process Adjustment 562
15.2 Process Adjustment Using Feedback Control 566
15.3 Excessive Adjustment Sometimes Required by MMSE Control 580
15.4 Minimum Cost Control with Fixed Costs of Adjustment and Monitoring 582
15.5 Feedforward Control 588
15.6 Monitoring Values of Parameters of Forecasting and Feedback Adjustment Schemes, 599
Appendix A5. 1 Feedback Control Schemes Where the Adjustment Variance Is Restricted, 600
Appendix A15. 2 Choice of the Sampling Interval 609
Exercises 613
Part Five Charts and Tables 617
Collection of Tables and Charts 619
Collection of Time Series Used for Examples in the Text and in Exercises 625
References 642
Index 659
1
Introduction
A time series is a sequence of observations taken sequentially in time. Many sets of data appear as time series: a monthly sequence of the quantity of goods shipped from a factory, a weekly series of the number of road accidents, daily rainfall amounts, hourly observations made on the yield of a chemical process, and so on. Examples of time series abound in such fields as economics, business, engineering, the natural sciences (especially geophysics and meteorology), and the social sciences. Examples of data of the kind that we will be concerned with are displayed as time series plots in Figures 2.1 and 4.1. An intrinsic feature of a time series is that, typically, adjacent observations are dependent. The nature of this dependence among observations of a time series is of considerable practical interest. Time series analysis is concerned with techniques for the analysis of this dependence. This requires the development of stochastic and dynamic models for time series data and the use of such models in important areas of application.
In the subsequent chapters of this book, we present methods for building, identifying, fitting, and checking models for time series and dynamic systems. The methods discussed are appropriate for discrete (sampled-data) systems, where observation of the system occurs at equally spaced intervals of time.
We illustrate the use of these time series and dynamic models in five important areas of application:
- The forecasting of future values of a time series from current and past values.
- The determination of the transfer function of a system subject to inertia-the determination of a dynamic input-output model that can show the effect on the output of a system of any given series of inputs.
- The use of indicator input variables in transfer function models to represent and assess the effects of unusual intervention events on the behavior of a time series.
- The examination of interrelationships among several related time series variables of interest and determination of appropriate multivariate dynamic models to represent these joint relationships among the variables over time.
- The design of simple control schemes by means of which potential deviations of the system output from a desired target may, so far as possible, be compensated by adjustment of the input series values.
1.1 Five Important Practical Problems
1.1.1 Forecasting Time Series
The use at time t of available observations from a time series to forecast its value at some future time t + l can provide a basis for (1) economic and business planning, (2) production planning, (3) inventory and production control, and (4) control and optimization of industrial processes. As originally described by Holt et al. (1963), Brown (1962), and the Imperial Chemical Industries (ICI) monograph on short term forecasting (Coutie, 1964), forecasts are usually needed over a period known as the lead time, which varies with each problem. For example, the lead time in the inventory control problem was defined by Harrison (1965) as a period that begins when an order to replenish stock is placed with the factory and lasts until the order is delivered into stock.
We will assume that observations are available at discrete, equispaced intervals of time. For example, in a sales forecasting problem, the sales zt in the current month t and the sales zt-1, zt-2, zt-3, ... in previous months might be used to forecast sales for lead times l = 1, 2, 3, ..., 12 months ahead. Denote by z^t(l) the forecast made at origin t of the sales zt+l at some future time t + l, that is, at lead time l. The function z^t(l), which provides the forecasts at origin t for all future lead times, based on the available information from the current and previous values zt, zt-1, zt-2, zt-3, ... through time t, will be called the forecast function at origin t. Our objective is to obtain a forecast function such that the mean square of the deviations zt+l-z^t(l) between the actual and forecasted values is as small as possible for each lead time l.
In addition to calculating the best forecasts, it is also necessary to specify their accuracy, so that, for example, the risks associated with decisions based upon the forecasts may be calculated. The accuracy of the forecasts may be expressed by calculating probability limits on either side of each forecast. These limits may be calculated for any convenient set of probabilities, for example, 50 and 95%. They are such that the realized value of the time series, when it eventually occurs, will be included within these limits with the stated probability. To illustrate, Figure 1.1 shows the last 20 values of a time series culminating at time t. Also shown are forecasts made from origin t for lead times l = 1, 2, ..., 13, together with the 50% probability limits.
Figure 1.1 Values of a time series with forecast function and 50% probability limits.
Methods for obtaining forecasts and estimating probability limits are discussed in detail in Chapter 5. These forecasting methods are developed based on the assumption that the time series zt follows a stochastic model of known form. Consequently, in Chapters 3 and 4 a useful class of such time series models that might be appropriate to represent the behavior of a series zt, called autoregressive integrated moving average (ARIMA) models, are introduced and many of their properties are studied. Subsequently, in Chapters 6, 7, and 8 the practical matter of how these models may be developed for actual time series data is explored, and the methods are described through the three-stage procedure of tentative model identification or specification, estimation of model parameters, and model checking and diagnostics.
1.1.2 Estimation of Transfer Functions
A topic of considerable industrial interest is the study of process dynamics discussed, for example, by Aström and Bohlin (1966, pp. 96-111) and Hutchinson and Shelton (1967). Such a study is made (1) to achieve better control of existing plants and (2) to improve the design of new plants. In particular, several methods have been proposed for estimating the transfer function of plant units from process records consisting of an input time series Xt and an output time series Yt. Sections of such records are shown in Figure 1.2, where the input Xt is the rate of air supply and the output Yt is the concentration of carbon dioxide produced in a furnace. The observations were made at 9-second intervals. A hypothetical impulse response function vj, j = 0, 1, 2, ..., which determines the transfer function for the system through a dynamic linear relationship between input Xt and output Yt of the form Yt=Sj=08vjXt-j, is also shown in the figure as a bar chart. Transfer function models that relate an input process Xt to an output process Yt are introduced in Chapter 11 and many of their properties are examined.
Figure 1.2 Input and output time series in relation to a dynamic system.
Methods for estimating transfer function models based on deterministic perturbations of the input, such as step, pulse, and sinusoidal changes, have not always been successful. This is because, for perturbations of a magnitude that are relevant and tolerable, the response of the system may be masked by uncontrollable disturbances referred to collectively as noise. Statistical methods for estimating transfer function models that make allowance for noise in the system are described in Chapter 12. The estimation of dynamic response is of considerable interest in economics, engineering, biology, and many other fields.
Another important application of transfer function models is in forecasting. If, for example, the dynamic relationship between two time series Yt and Xt can be determined, past values of both series may be used in forecasting Yt. In some situations, this approach can lead to a considerable reduction in the errors of the forecasts.
1.1.3 Analysis of Effects of Unusual Intervention Events to a System
In some situations, it may be known that certain exceptional external events, intervention events, could have affected the time series zt under study. Examples of such intervention events include the incorporation of new environmental regulations, economic policy changes, strikes, and special promotion campaigns. Under such circumstances, we may use transfer function models, as discussed in Section 1.1.2, to account for the effects of the intervention event on the series zt, but where the "input" series will be in the form of a simple indicator variable taking only the values 1 and 0 to indicate (qualitatively) the presence or absence of the event.
In these cases, the intervention analysis is undertaken to obtain a quantitative measure of the impact of the intervention event on the time series of interest. For example, Box and Tiao (1975) used...
