Linear Models and Time-Series Analysis: Regression, Anova, Arma and Garch

Preface xiii Part I Linear Models: Regression and ANOVA 1 1 The Linear Model 3 1.1 Regression, Correlation, and Causality 3 1.2 Ordinary and Generalized Least Squares 7 1.2.1 Ordinary Least Squares Estimation 7 1.2.2 Further Aspects of Regression and OLS 8 1.2.3 Generalized Least Squares 12 1.3 The Geometric Approach to Least Squares 17 1.3.1 Projection 17 1.3.2 Implementation 22 1.4 Linear Parameter Restrictions 26 1.4.1 Formulation and Estimation 27 1.4.2 Estimability and Identifiability 30 1.4.3 Moments and the Restricted GLS Estimator 32 1.4.4 Testing With h = 0 34 1.4.5 Testing With Nonzero h 37 1.4.6 Examples 37 1.4.7 Confidence Intervals 42 1.5 Alternative Residual Calculation 47 1.6 Further Topics 51 1.7 Problems 56 1.A Appendix: Derivation of the BLUS Residual Vector 60 1.B Appendix: The Recursive Residuals 64 1.C Appendix: Solutions 66 2 Fixed Effects ANOVA Models 77 2.1 Introduction: Fixed, Random, and Mixed Effects Models 77 2.2 Two Sample t-Tests for Differences in Means 78 2.3 The Two Sample t-Test with Ignored Block Effects 84 2.4 One-Way ANOVA with Fixed Effects 87 2.4.1 The Model 87 2.4.2 Estimation and Testing 88 2.4.3 Determination of Sample Size 91 2.4.4 The ANOVA Table 93 2.4.5 Computing Confidence Intervals 97 2.4.6 A Word on Model Assumptions 103 2.5 Two-Way Balanced Fixed Effects ANOVA 107 2.5.1 The Model and Use of the Interaction Terms 107 2.5.2 Sums of Squares Decomposition without Interaction 108 2.5.3 Sums of Squares Decomposition with Interaction 113 2.5.4 Example and Codes 117 3 Introduction to Random and Mixed Effects Models 127 3.1 One-Factor Balanced Random Effects Model 128 3.1.1 Model and Maximum Likelihood Estimation 128 3.1.2 Distribution Theory and ANOVA Table 131 3.1.3 Point Estimation, Interval Estimation, and Significance Testing 137 3.1.4 Satterthwaite's Method 139 3.1.5 Use of SAS 142 3.1.6 Approximate Inference in the Unbalanced Case 143 3.1.6.1 Point Estimation in the Unbalanced Case 144 3.1.6.2 Interval Estimation in the Unbalanced Case 150 3.2 Crossed Random Effects Models 152 3.2.1 Two Factors 154 3.2.1.1 With Interaction Term 154 3.2.1.2 Without Interaction Term 157 3.2.2 Three Factors 157 3.3 Nested Random Effects Models 162 3.3.1 Two Factors 162 3.3.1.1 Both Effects Random: Model and Parameter Estimation 162 3.3.1.2 Both Effects Random: Exact and Approximate Confidence Intervals 167 3.3.1.3 Mixed Model Case 170 3.3.2 Three Factors 174 3.3.2.1 All Effects Random 174 3.3.2.2 Mixed: Classes Fixed 176 3.3.2.3 Mixed: Classes and Subclasses Fixed 177 3.4 Problems 177 3.A Appendix: Solutions 178 Part II Time-Series Analysis: ARMAX Processes 185 4 The AR(1) Model 187 4.1 Moments and Stationarity 188 4.2 Order of Integration and Long-Run Variance 195 4.3 Least Squares and ML Estimation 196 4.3.1 OLS Estimator of a 196 4.3.2 Likelihood Derivation I 196 4.3.3 Likelihood Derivation II 198 4.3.4 Likelihood Derivation III 198 4.3.5 Asymptotic Distribution 199 4.4 Forecasting 200 4.5 Small Sample Distribution of the OLS and ML Point Estimators 204 4.6 Alternative Point Estimators of a 208 4.6.1 Use of the Jackknife for Bias Reduction 208 4.6.2 Use of the Bootstrap for Bias Reduction 209 4.6.3 Median-Unbiased Estimator 211 4.6.4 Mean-Bias Adjusted Estimator 211 4.6.5 Mode-Adjusted Estimator 212 4.6.6 Comparison 213 4.7 Confidence Intervals for a 215 4.8 Problems 219 5 Regression Extensions: AR(1) Errors and Time-varying Parameters 223 5.1 The AR(1) Regression Model and the Likelihood 223 5.2 OLS Point and Interval Estimation of a 225 5.3 Testing a = 0 in the ARX(1) Model 229 5.3.1 Use of Confidence Intervals 229 5.3.2 The Durbin-Watson Test 229 5.3.3 Other Tests for First-order Autocorrelation 231 5.3.4 Further Details on the Durbin-Watson Test 236 5.3.4.1 The Bounds Test, and Critique of Use of p-Values 236 5.3.4.2 Limiting Power as a --> ±1 239 5.4 Bias-Adjusted Point Estimation 243 5.5 Unit Root Testing in the ARX(1) Model 246 5.5.1 Null is a = 1 248 5.5.2 Null is a < 1 256 5.6 Time-Varying Parameter Regression 259 5.6.1 Motivation and Introductory Remarks 260 5.6.2 The Hildreth-Houck Random Coeffcient Model 261 5.6.3 The TVP Random Walk Model 269 5.6.3.1 Covariance Structure and Estimation 271 5.6.3.2 Testing for Parameter Constancy 274 5.6.4 Rosenberg Return to Normalcy Model 277 6 Autoregressive and Moving Average Processes 281 6.1 AR(p) Processes 281 6.1.1 Stationarity and Unit Root Processes 282 6.1.2 Moments 284 6.1.3 Estimation 287 6.1.3.1 Without Mean Term 287 6.1.3.2 Starting Values 290 6.1.3.3 With Mean Term 292 6.1.3.4 Approximate Standard Errors 293 6.2 Moving Average Processes 294 6.2.1 MA(1) Process 294 6.2.2 MA(q) Processes 299 6.3 Problems 301 6.A Appendix: Solutions 302 7 ARMA Processes 311 7.1 Basics of ARMA Models 311 7.1.1 The Model 311 7.1.2 Zero Pole Cancellation 312 7.1.3 Simulation 313 7.1.4 The ARIMA(p, d, q) Model 314 7.2 Infinite AR and MA Representations 315 7.3 Initial Parameter Estimation 317 7.3.1 Via the Infinite AR Representation 318 7.3.2 Via Infinite AR and Ordinary Least Squares 318 7.4 Likelihood-Based Estimation 322 7.4.1 Covariance Structure 322 7.4.2 Point Estimation 324 7.4.3 Interval Estimation 328 7.4.4 Model Mis-specification 330 7.5 Forecasting 331 7.5.1 AR(p) Model 331 7.5.2 MA(q) and ARMA(p, q) Models 335 7.5.3 ARIMA(p, d, q) Models 339 7.6 Bias-Adjusted Point Estimation: Extension to the ARMAX(1, q) model 339 7.7 Some ARIMAX Model Extensions 343 7.7.1 Stochastic Unit Root 344 7.7.2 Threshold Autoregressive Models 346 7.7.3 Fractionally Integrated ARMA (ARFIMA) 347 7.8 Problems 349 7.A Appendix: Generalized Least Squares for ARMA Estimation 351 7.B Appendix: Multivariate AR(p) Processes and Stationarity, and General Block Toeplitz Matrix Inversion 357 8 Correlograms 359 8.1 Theoretical and Sample Autocorrelation Function 359 8.1.1 Definitions 359 8.1.2 Marginal Distributions 365 8.1.3 Joint Distribution 371 8.1.3.1 Support 371 8.1.3.2 Asymptotic Distribution 372 8.1.3.3 Small-Sample Joint Distribution Approximation 375 8.1.4 Conditional Distribution Approximation 381 8.2 Theoretical and Sample Partial Autocorrelation Function 384 8.2.1 Partial Correlation 384 8.2.2 Partial Autocorrelation Function 389 8.2.2.1 TPACF: First Definition 389 8.2.2.2 TPACF: Second Definition 390 8.2.2.3 Sample Partial Autocorrelation Function 392 8.3 Problems 396 8.A Appendix: Solutions 397 9 ARMA Model Identification 405 9.1 Introduction 405 9.2 Visual Correlogram Analysis 407 9.3 Significance Tests 412 9.4 Penalty Criteria 417 9.5 Use of the Conditional SACF for Sequential Testing 421 9.6 Use of the Singular Value Decomposition 436 9.7 Further Methods: Pattern Identification 439 Part III Modeling Financial Asset Returns 443 10 Univariate GARCH Modeling 445 10.1 Introduction 445 10.2 Gaussian GARCH and Estimation 450 10.2.1 Basic Properties 451 10.2.2 Integrated GARCH 452 10.2.3 Maximum Likelihood Estimation 453 10.2.4 Variance Targeting Estimator 459 10.3 Non-Gaussian ARMA-APARCH, QMLE, and Forecasting 459 10.3.1 Extending the Volatility, Distribution, and Mean Equations 459 10.3.2 Model Mis-specification and QMLE 464 10.3.3 Forecasting 467 10.4 Near-Instantaneous Estimation of NCT-APARCH(1,1) 468 10.5 S alpha,beta-APARCH and Testing the IID Stable Hypothesis 473 10.6 Mixed Normal GARCH 477 10.6.1 Introduction 477 10.6.2 The MixN(k)-GARCH(r, s) Model 478 10.6.3 Parameter Estimation and Model Features 479 10.6.4 Time-Varying Weights 482 10.6.5 Markov Switching Extension 484 10.6.6 Multivariate Extensions 484 11 Risk Prediction and Portfolio Optimization 487 11.1 Value at Risk and Expected Shortfall Prediction 487 11.2 MGARCH Constructs Via Univariate GARCH 493 11.2.1 Introduction 493 11.2.2 The Gaussian CCC and DCC Models 494 11.2.3 Morana Semi-Parametric DCC Model 497 11.2.4 The COMFORT Class 499 11.2.5 Copula Constructions 503 11.3 Introducing Portfolio Optimization 504 11.3.1 Some Trivial Accounting 504 11.3.2 Markowitz and DCC 510 11.3.3 Portfolio Optimization Using Simulation 513 11.3.4 The Univariate Collapsing Method 516 11.3.5 The ES Span 521 12 Multivariate t Distributions 525 12.1 Multivariate Student's t 525 12.2 Multivariate Noncentral Student's t 530 12.3 Jones Multivariate t Distribution 534 12.4 Shaw and Lee Multivariate t Distributions 538 12.5 The Meta-Elliptical t Distribution 540 12.5.1 The FaK Distribution 541 12.5.2 The AFaK Distribution 542 12.5.3 FaK and AFaK Estimation: Direct Likelihood Optimization 546 12.5.4 FaK and AFaK Estimation: Two-Step Estimation 548 12.5.5 Sums of Margins of the AFaK 555 12.6 MEST: Marginally Endowed Student's t 556 12.6.1 SMESTI Distribution 557 12.6.2 AMESTI Distribution 558 12.6.3 MESTI Estimation 561 12.6.4 AoNm-MEST 564 12.6.5 MEST Distribution 573 12.7 Some Closing Remarks 574 12.A ES of Convolution of AFaK Margins 575 12.B Covariance Matrix for the FaK 581 13 Weighted Likelihood 587 13.1 Concept 587 13.2 Determination of Optimal Weighting 592 13.3 Density Forecasting and Backtest Overfitting 594 13.4 Portfolio Optimization Using (A)FaK 600 14 Multivariate Mixture Distributions 611 14.1 The Mixk Nd Distribution 611 14.1.1 Density and Simulation 612 14.1.2 Motivation for Use of Mixtures 612 14.1.3 Quasi-Bayesian Estimation and Choice of Prior 614 14.1.4 Portfolio Distribution and Expected Shortfall 620 14.2 Model Diagnostics and Forecasting 623 14.2.1 Assessing Presence of a Mixture 623 14.2.2 Component Separation and Univariate Normality 625 14.2.3 Component Separation and Multivariate Normality 629 14.2.4 Mixed Normal Weighted Likelihood and Density Forecasting 631 14.2.5 Density Forecasting: Optimal Shrinkage 633 14.2.6 Moving Averages of lambda 640 14.3 MCD for Robustness and Mix2Nd Estimation 645 14.4 Some Thoughts on Model Assumptions and Estimation 647 14.5 The Multivariate Laplace and Mixk Lapd Distributions 649 14.5.1 The Multivariate Laplace and EM Algorithm 650 14.5.2 The Mixk Lapd and EM Algorithm 654 14.5.3 Estimation via MCD Split and Forecasting 658 14.5.4 Estimation of Parameter b 660 14.5.5 Portfolio Distribution and Expected Shortfall 662 14.5.6 Fast Evaluation of the Bessel Function 663 Part IV Appendices 667 Appendix A Distribution of Quadratic Forms 669 A.1 Distribution and Moments 669 A.1.1 Probability Density and Cumulative Distribution Functions 669 A.1.2 Positive Integer Moments 671 A.1.3 Moment Generating Functions 673 A.2 Basic Distributional Results 677 A.3 Ratios of Quadratic Forms in Normal Variables 679 A.3.1 Calculation of the CDF 680 A.3.2 Calculation of the PDF 681 A.3.2.1 Numeric Differentiation 682 A.3.2.2 Use of Geary's formula 682 A.3.2.3 Use of Pan's Formula 683 A.3.2.4 Saddlepoint Approximation 685 A.4 Problems 689 A.A Appendix: Solutions 690 Appendix B Moments of Ratios of Quadratic Forms 695 B.1 For X ~ Nn(0, 2I) and B = I 695 B.2 For X ~ N(0, Sigma) 708 B.3 For X ~ N(my, I) 713 B.4 For X ~ N(my, Sigma) 720 B.5 Useful Matrix Algebra Results 725 B.6 Saddlepoint Equivalence Result 729 Appendix C Some Useful Multivariate Distribution Theory 733 C.1 Student's t Characteristic Function 733 C.2 Sphericity and Ellipticity 739 C.2.1 Introduction 739 C.2.2 Sphericity 740 C.2.3 Ellipticity 748 C.2.4 Testing Ellipticity 768 Appendix D Introducing the SAS Programming Language 773 D.1 Introduction to SAS 774 D.1.1 Background 774 D.1.2 Working with SAS on a PC 775 D.1.3 Introduction to the Data Step and the Program Data Vector 777 D.2 Basic Data Handling 783 D.2.1 Method 1 784 D.2.2 Method 2 785 D.2.3 Method 3 786 D.2.4 Creating Data Sets from Existing Data Sets 787 D.2.5 Creating Data Sets from Procedure Output 788 D.3 Advanced Data Handling 790 D.3.1 String Input and Missing Values 790 D.3.2 Using set with first.var and last.var 791 D.3.3 Reading in Text Files 795 D.3.4 Skipping over Headers 796 D.3.5 Variable and Value Labels 796 D.4 Generating Charts, Tables, and Graphs 797 D.4.1 Simple Charting and Tables 798 D.4.2 Date and Time Formats/Informats 801 D.4.3 High Resolution Graphics 803 D.4.3.1 The GPLOT Procedure 803 D.4.3.2 The GCHART Procedure 805 D.4.4 Linear Regression and Time-Series Analysis 806 D.5 The SAS Macro Processor 809 D.5.1 Introduction 809 D.5.2 Macro Variables 810 D.5.3 Macro Programs 812 D.5.4 A Useful Example 814 D.5.4.1 Method 1 814 D.5.4.2 Method 2 816 D.6 Problems 817 D.7 Appendix: Solutions 819 Bibliography 825 Index 875