Matrix Algebra Useful for Statistics

PREFACE xvii PREFACE TO THE FIRST EDITION xix INTRODUCTION xxi ABOUT THE COMPANION WEBSITE xxxi PART I DEFINITIONS, BASIC CONCEPTS, AND MATRIX OPERATIONS 1 1 Vector Spaces, Subspaces, and Linear Transformations 3 1.1 Vector Spaces 3 1.2 Base of a Vector Space 5 1.3 Linear Transformations 7 2 Matrix Notation and Terminology 11 2.1 Plotting of a Matrix 14 2.2 Vectors and Scalars 16 2.3 General Notation 16 3 Determinants 21 3.1 Expansion by Minors 21 3.2 Formal Definition 25 3.3 Basic Properties 27 3.4 Elementary Row Operations 34 3.5 Examples 37 3.6 Diagonal Expansion 39 3.7 The Laplace Expansion 42 3.8 Sums and Differences of Determinants 44 3.9 A Graphical Representation of a 3 × 3 Determinant 45 4 Matrix Operations 51 4.1 The Transpose of a Matrix 51 4.2 Partitioned Matrices 52 4.3 The Trace of a Matrix 55 4.4 Addition 56 4.5 Scalar Multiplication 58 4.6 Equality and the Null Matrix 58 4.7 Multiplication 59 4.8 The Laws of Algebra 74 4.9 Contrasts With Scalar Algebra 76 4.10 Direct Sum of Matrices 77 4.11 Direct Product of Matrices 78 4.12 The Inverse of a Matrix 80 4.13 Rank of a Matrix--Some Preliminary Results 82 4.14 The Number of LIN Rows and Columns in a Matrix 84 4.15 Determination of the Rank of a Matrix 85 4.16 Rank and Inverse Matrices 87 4.17 Permutation Matrices 87 5 Special Matrices 97 5.1 Symmetric Matrices 97 5.2 Matrices Having All Elements Equal 102 5.3 Idempotent Matrices 104 5.4 Orthogonal Matrices 106 5.5 Parameterization of Orthogonal Matrices 109 5.6 Quadratic Forms 110 5.7 Positive Definite Matrices 113 6 Eigenvalues and Eigenvectors 119 6.1 Derivation of Eigenvalues 119 6.2 Elementary Properties of Eigenvalues 122 6.3 Calculating Eigenvectors 125 6.4 The Similar Canonical Form 128 6.5 Symmetric Matrices 131 6.6 Eigenvalues of Orthogonal and Idempotent Matrices 135 6.7 Eigenvalues of Direct Products and Direct Sums of Matrices 138 6.8 Nonzero Eigenvalues of AB and BA 140 7 Diagonalization of Matrices 145 7.1 Proving the Diagonability Theorem 145 7.2 Other Results for Symmetric Matrices 148 7.3 The Cayley-Hamilton Theorem 152 7.4 The Singular-Value Decomposition 153 8 Generalized Inverses 159 8.1 The Moore-Penrose Inverse 159 8.2 Generalized Inverses 160 8.3 Other Names and Symbols 164 8.4 Symmetric Matrices 165 9 Matrix Calculus 171 9.1 Matrix Functions 171 9.2 Iterative Solution of Nonlinear Equations 174 9.3 Vectors of Differential Operators 175 9.4 Vec and Vech Operators 179 9.5 Other Calculus Results 181 9.6 Matrices with Elements That Are Complex Numbers 188 9.7 Matrix Inequalities 189 PART II APPLICATIONS OF MATRICES IN STATISTICS 199 10 Multivariate Distributions and Quadratic Forms 201 10.1 Variance-Covariance Matrices 202 10.2 Correlation Matrices 203 10.3 Matrices of Sums of Squares and Cross-Products 204 10.4 The Multivariate Normal Distribution 207 10.5 Quadratic Forms and 2-Distributions 208 10.6 Computing the Cumulative Distribution Function of a Quadratic Form 213 11 Matrix Algebra of Full-Rank Linear Models 219 11.1 Estimation of by the Method of Least Squares 220 11.2 Statistical Properties of the Least-Squares Estimator 226 11.3 Multiple Correlation Coefficient 229 11.4 Statistical Properties under the Normality Assumption 231 11.5 Analysis of Variance 233 11.6 The Gauss-Markov Theorem 234 11.7 Testing Linear Hypotheses 237 11.8 Fitting Subsets of the x-Variables 246 11.9 The Use of the R(.|.) Notation in Hypothesis Testing 247 12 Less-Than-Full-Rank Linear Models 253 12.1 General Description 253 12.2 The Normal Equations 256 12.3 Solving the Normal Equations 257 12.4 Expected Values and Variances 259 12.5 Predicted y-Values 260 12.6 Estimating the Error Variance 261 12.7 Partitioning the Total Sum of Squares 262 12.8 Analysis of Variance 263 12.9 The R(s|s) Notation 265 12.10 Estimable Linear Functions 266 12.11 Confidence Intervals 272 12.12 Some Particular Models 272 12.13 The R(s|s) Notation (Continued) 277 12.14 Reparameterization to a Full-Rank Model 281 13 Analysis of Balanced Linear Models Using Direct Products of Matrices 287 13.1 General Notation for Balanced Linear Models 289 13.2 Properties Associated with Balanced Linear Models 293 13.3 Analysis of Balanced Linear Models 298 14 Multiresponse Models 313 14.1 Multiresponse Estimation of Parameters 314 14.2 Linear Multiresponse Models 316 14.3 Lack of Fit of a Linear Multiresponse Model 318 PART III MATRIX COMPUTATIONS AND RELATED SOFTWARE 327 15 SAS/IML 329 15.1 Getting Started 329 15.2 Defining a Matrix 329 15.3 Creating a Matrix 330 15.4 Matrix Operations 331 15.5 Explanations of SAS Statements Used Earlier in the Text 354 16 Use of MATLAB in Matrix Computations 363 16.1 Arithmetic Operators 363 16.2 Mathematical Functions 364 16.3 Construction of Matrices 365 16.4 Two- and Three-Dimensional Plots 371 17 Use of R in Matrix Computations 383 17.1 Two- and Three-Dimensional Plots 396 Exercises 408 APPENDIX 413 INDEX 475