Applied Linear Regression

1 Scatterplots 1 1.1 Scatterplots 2 1.2 Mean Functions 9 1.3 Variance Functions 12 1.4 Summary Graph 12 1.5 Tools for Looking at Scatterplots 13 1.6 Scatterplot Matrices 15 1.7 Problems 17 2 Simple Linear Regression 21 2.1 Ordinary Least Squares Estimation 22 2.2 Least Squares Criterion 24 2.3 Estimating the Variance sigma2 26 2.4 Properties of Least Squares Estimates 27 2.5 Estimated Variances 28 2.6 Confidence Intervals and t-Tests 29 2.7 The Coefficient of Determination, R2 33 2.8 The Residuals 35 2.9 Problems 37 3 Multiple Regression 49 3.1 Adding a Regressor to a Simple Linear Regression Model 49 3.2 The Multiple Linear Regression Model 53 3.3 Predictors and Regressors 53 3.4 Ordinary Least Squares 57 3.5 Predictions, Fitted Values and Linear Combinations 65 3.6 Problems 66 4 Interpretation of Main Effects 71 4.1 Understanding Parameter Estimates 71 4.2 Dropping Regressors 81 4.3 Experimentation Versus Observation 84 4.4 Sampling from a Normal Population 86 4.5 More on R2 88 4.6 Problems 90 5 Complex Regressors 95 5.1 Factors 95 5.2 Many Factors 105 5.3 Polynomial Regression 106 5.4 Splines 109 5.5 Principal Components 112 5.6 Missing Data 115 5.7 Problems 118 6 Testing and Analysis of Variance 129 6.1 F-tests 130 6.2 The Analysis of Variance 134 6.3 Comparisons of Means 138 6.4 Power and Non-null Distributions 138 6.5 Wald Tests 140 6.6 Interpreting Tests 142 6.7 Problems 145 7 Variances 151 7.1 Weighted Least Squares 151 7.2 Misspecified Variances 157 7.3 General Correlation Structures 162 7.4 Mixed Models 163 7.5 Variance Stabilizing Transformations 165 7.6 The Delta Method 166 7.7 The Bootstrap 168 7.8 Problems 173 8 Transformations 179 8.1 Transformation Basics 179 8.2 A General Approach to Transformations 185 8.3 Transforming the Response 190 8.4 Transformations of Nonpositive Variables 192 8.5 Additive Models 192 8.6 Problems 193 9 Regression Diagnostics 199 9.1 The Residuals 199 9.2 Testing for Curvature 206 9.3 Nonconstant Variance 208 9.4 Outliers 208 9.5 Influence of Cases 212 9.6 Normality Assumption 218 9.7 Problems 220 10 Variable Selection 227 10.1 Variable Selection and Parameter Assessment 228 10.2 Variable Selection for Discovery 230 10.3 Model Selection for Prediction 238 10.4 Problems 241 11 Nonlinear Regression 245 11.1 Estimation for Nonlinear Mean Functions 246 11.2 Inference Assuming Large Samples 249 11.3 Starting Values 249 11.4 Bootstrap Inference 255 11.5 Further Reading 257 11.6 Problems 258 12 Binomial and Poisson Regression 263 12.1 Distributions for Counted Data 263 12.2 Regression Models For Counts 265 12.3 Poisson Regression 271 12.4 Transferring What You Know about Linear Models 276 12.5 Generalized Linear Models 278 12.6 Problems 278 A Appendix 283 A.1 Website 283 A.2 Means, Variances, Covariances and Correlations 283 A.3 Least Squares for Simple Regression 286 A.4 Means and Variances of Least Squares Estimates 286 A.5 Estimating E(Y |X) using a Smoother 288 A.6 A Brief Introduction to Matrices and Vectors 290 A.7 Random Vectors 295 A.8 Least Squares Using Matrices 295 A.9 The QR factorization 299 A.10 Spectral Decomposition 300 A.11 Maximum Likelihood Estimates 300 A.12 The Box-Cox Method for Transformations 302 A.13 Case Deletion in Linear Regression 305 Bibliography 321 Index 322