Scalar, Vector, and Matrix Mathematics

Preface to the Revised and Expanded Edition xvii
Preface to the Second Edition xix
Preface to the First Edition xxi
Special Symbols xxv
Conventions, Notation, and Terminology xxxvii
1. Sets, Logic, Numbers, Relations, Orderings, Graphs, and Functions 1
1.1 Sets 1
1.2 Logic 2
1.3 Relations and Orderings 5
1.4 Directed and Symmetric Graphs 9
1.5 Numbers 12
1.6 Functions and Their Inverses 16
1.7 Facts on Logic 21
1.8 Facts on Sets 22
1.9 Facts on Graphs 25
1.10 Facts on Functions 26
1.11 Facts on Integers 28
1.12 Facts on Finite Sums 36
1.13 Facts on Factorials 49
1.14 Facts on Finite Products 52
1.15 Facts on Numbers 52
1.16 Facts on Binomial Coefficients 54
1.17 Facts on Fibonacci, Lucas, and Pell Numbers 95
1.18 Facts on Arrangement, Derangement, and Catalan Numbers 103
1.19 Facts on Cycle, Subset, Eulerian, Bell, and Ordered Bell Numbers 105
1.20 Facts on Partition Numbers, the Totient Function, and Divisor Sums 113
1.21 Facts on Convex Functions 116
1.22 Notes 118
2. Equalities and Inequalities 119
2.1 Facts on Equalities and Inequalities in One Variable 119
2.2 Facts on Equalities and Inequalities in Two Variables 129
2.3 Facts on Equalities and Inequalities in Three Variables 146
2.4 Facts on Equalities and Inequalities in Four Variables 177
2.5 Facts on Equalities and Inequalities in Five Variables 183
2.6 Facts on Equalities and Inequalities in Six Variables 184
2.7 Facts on Equalities and Inequalities in Seven Variables 186
2.8 Facts on Equalities and Inequalities in Eight Variables 187
2.9 Facts on Equalities and Inequalities in Nine Variables 187
2.10 Facts on Equalities and Inequalities in Sixteen Variables 187
2.11 Facts on Equalities and Inequalities in n Variables 188
2.12 Facts on Equalities and Inequalities in 2n Variables 215
2.13 Facts on Equalities and Inequalities in 3n Variables 226
2.14 Facts on Equalities and Inequalities in 4n Variables 226
2.15 Facts on Equalities and Inequalities for the Logarithm Function 226
2.16 Facts on Equalities for Trigonometric Functions 231
2.17 Facts on Inequalities for Trigonometric Functions 246
2.18 Facts on Equalities and Inequalities for Inverse Trigonometric Functions 254
2.19 Facts on Equalities and Inequalities for Hyperbolic Functions 261
2.20 Facts on Equalities and Inequalities for Inverse Hyperbolic Functions 264
2.21 Facts on Equalities and Inequalities in Complex Variables 266
2.22 Notes 276
3. Basic Matrix Properties 277
3.1 Vectors 277
3.2 Matrices 280
3.3 Transpose and Inner Product 285
3.4 Geometrically Defined Sets 290
3.5 Range and Null Space 290
3.6 Rank and Defect 292
3.7 Invertibility 294
3.8 The Determinant 299
3.9 Partitioned Matrices 302
3.10 Majorization 305
3.11 Facts on One Set 306
3.12 Facts on Two or More Sets 310
3.13 Facts on Range, Null Space, Rank, and Defect 315
3.14 Facts on the Range, Rank, Null Space, and Defect of Partitioned Matrices 320
3.15 Facts on the Inner Product, Outer Product, Trace, and Matrix Powers 326
3.16 Facts on the Determinant 329
3.17 Facts on the Determinant of Partitioned Matrices 334
3.18 Facts on Left and Right Inverses 342
3.19 Facts on the Adjugate 345
3.20 Facts on the Inverse 348
3.21 Facts on Bordered Matrices 351
3.22 Facts on the Inverse of Partitioned Matrices 352
3.23 Facts on Commutators 354
3.24 Facts on Complex Matrices 356
3.25 Facts on Majorization 359
3.26 Notes 362
4. Matrix Classes and Transformations 363
4.1 Types of Matrices 363
4.2 Matrices Related to Graphs 367
4.3 Lie Algebras 368
4.4 Abstract Groups 369
4.5 Addition Groups 371
4.6 Multiplication Groups 371
4.7 Matrix Transformations 373
4.8 Projectors, Idempotent Matrices, and Subspaces 374
4.9 Facts on Elementary, Group-Invertible, Range-Hermitian, Range-Disjoint, and Range-Spanning Matrices 376
4.10 Facts on Normal, Hermitian, and Skew-Hermitian Matrices 377
4.11 Facts on Linear Interpolation 383
4.12 Facts on the Cross Product 384
4.13 Facts on Inner, Unitary, and Shifted-Unitary Matrices 387
4.14 Facts on Rotation Matrices 391
4.15 Facts on One Idempotent Matrix 396
4.16 Facts on Two or More Idempotent Matrices 398
4.17 Facts on One Projector 407
4.18 Facts on Two or More Projectors 409
4.19 Facts on Reflectors 416
4.20 Facts on Involutory Matrices 417
4.21 Facts on Tripotent Matrices 417
4.22 Facts on Nilpotent Matrices 418
4.23 Facts on Hankel and Toeplitz Matrices 420
4.24 Facts on Tridiagonal Matrices 422
4.25 Facts on Triangular, Hessenberg, and Irreducible Matrices 424
4.26 Facts on Matrices Related to Graphs 426
4.27 Facts on Dissipative, Contractive, Cauchy, and Centrosymmetric Matrices 427
4.28 Facts on Hamiltonian and Symplectic Matrices 427
4.29 Facts on Commutators 428
4.30 Facts on Partial Orderings 430
4.31 Facts on Groups 432
4.32 Facts on Quaternions 437
4.33 Notes 440
5. Geometry 441
5.1 Facts on Angles, Lines, and Planes 441
5.2 Facts on Triangles 443
5.3 Facts on Polygons and Polyhedra 489
5.4 Facts on Polytopes 493
5.5 Facts on Circles, Ellipses, Spheres, and Ellipsoids 495
6. Polynomial Matrices and Rational Transfer Functions 499
6.1 Polynomials 499
6.2 Polynomial Matrices 501
6.3 The Smith Form and Similarity Invariants 503
6.4 Eigenvalues 506
6.5 Eigenvectors 511
6.6 The Minimal Polynomial 512
6.7 Rational Transfer Functions and the Smith-McMillan Form 513
6.8 Facts on Polynomials and Rational Functions 517
6.9 Facts on the Characteristic and Minimal Polynomials 524
6.10 Facts on the Spectrum 530
6.11 Facts on Graphs and Nonnegative Matrices 537
6.12 Notes 544
7. Matrix Decompositions 545
7.1 Smith Decomposition 545
7.2 Reduced Row Echelon Decomposition 545
7.3 Multicompanion and Elementary Multicompanion Decompositions 546
7.4 Jordan Decomposition 549
7.5 Schur Decomposition 553
7.6 Singular Value Decomposition, Polar Decomposition, and Full-Rank Factorization 555
7.7 Eigenstructure Properties 558
7.8 Pencils and the Kronecker Canonical Form 563
7.9 Facts on the Inertia 565
7.10 Facts on Matrix Transformations for One Matrix 569
7.11 Facts on Matrix Transformations for Two or More Matrices 575
7.12 Facts on Eigenvalues and Singular Values for One Matrix 579
7.13 Facts on Eigenvalues and Singular Values for Two or More Matrices 589
7.14 Facts on Matrix Pencils 597
7.15 Facts on Eigenstructure for One Matrix 597
7.16 Facts on Eigenstructure for Two or More Matrices 603
7.17 Facts on Matrix Factorizations 605
7.18 Facts on Companion, Vandermonde, Circulant, Permutation, and Hadamard Matrices 610
7.19 Facts on Simultaneous Transformations 617
7.20 Facts on Additive Decompositions 618
7.21 Notes 619
8. Generalized Inverses 621
8.1 Moore-Penrose Generalized Inverse 621
8.2 Drazin Generalized Inverse 625
8.3 Facts on the Moore-Penrose Generalized Inverse for One Matrix 628
8.4 Facts on the Moore-Penrose Generalized Inverse for Two or More Matrices 632
8.5 Facts on the Moore-Penrose Generalized Inverse for Range-Hermitian, Range-Disjoint, and Range-Spanning Matrices 641
8.6 Facts on the Moore-Penrose Generalized Inverse for Normal Matrices, Hermitian Matrices, and Partial Isometries 649
8.7 Facts on the Moore-Penrose Generalized Inverse for Idempotent Matrices 650
8.8 Facts on the Moore-Penrose Generalized Inverse for Projectors 652
8.9 Facts on the Moore-Penrose Generalized Inverse for Partitioned Matrices 659
8.10 Facts on the Drazin and Group Generalized Inverses for One Matrix 669
8.11 Facts on the Drazin and Group Generalized Inverses for Two or More Matrices 674
8.12 Facts on the Drazin and Group Generalized Inverses for Partitioned Matrices 678
8.13 Notes 679
9. Kronecker and Schur Algebra 681
9.1 Kronecker Product 681
9.2 Kronecker Sum and Linear Matrix Equations 683
9.3 Schur Product 685
9.4 Facts on the Kronecker Product 685
9.5 Facts on the Kronecker Sum 691
9.6 Facts on the Schur Product 697
9.7 Notes 701
10.Positive-Semidefinite Matrices 703
10.1 Positive-Semidefinite and Positive-Definite Orderings 703
10.2 Submatrices and Schur Complements 704
10.3 Simultaneous Diagonalization 707
10.4 Eigenvalue Inequalities 709
10.5 Exponential, Square Root, and Logarithm of Hermitian Matrices 713
10.6 Matrix Inequalities 714
10.7 Facts on Range and Rank 722
10.8 Facts on Unitary Matrices and the Polar Decomposition 723
10.9 Facts on Structured Positive-Semidefinite Matrices 724
10.10 Facts on Equalities and Inequalities for One Matrix 730
10.11 Facts on Equalities and Inequalities for Two or More Matrices 735
10.12 Facts on Equalities and Inequalities for Partitioned Matrices 749
10.13 Facts on the Trace for One Matrix 761
10.14 Facts on the Trace for Two or More Matrices 763
10.15 Facts on the Determinant for One Matrix 774
10.16 Facts on the Determinant for Two or More Matrices 776
10.17 Facts on Convex Sets and Convex Functions 785
10.18 Facts on Quadratic Forms for One Matrix 792
10.19 Facts on Quadratic Forms for Two or More Matrices 795
10.20 Facts on Simultaneous Diagonalization 799
10.21 Facts on Eigenvalues and Singular Values for One Matrix 800
10.22 Facts on Eigenvalues and Singular Values for Two or More Matrices 804
10.23 Facts on Alternative Partial Orderings 813
10.24 Facts on Generalized Inverses 815
10.25 Facts on the Kronecker and Schur Products 820
10.26 Notes 831
11.Norms 833
11.1 Vector Norms 833
11.2 Matrix Norms 835
11.3 Compatible Norms 838
11.4 Induced Norms 841
11.5 Induced Lower Bound 845
11.6 Singular Value Inequalities 847
11.7 Facts on Vector Norms 849
11.8 Facts on Vector p-Norms 853
11.9 Facts on Matrix Norms for One Matrix 860
11.10 Facts on Matrix Norms for Two or More Matrices 868
11.11 Facts on Matrix Norms for Commutators 884
11.12 Facts on Matrix Norms for Partitioned Matrices 885
11.13 Facts on Matrix Norms and Eigenvalues for One Matrix 890
11.14 Facts on Matrix Norms and Eigenvalues for Two or More Matrices 892
11.15 Facts on Matrix Norms and Singular Values for One Matrix 895
11.16 Facts on Matrix Norms and Singular Values for Two or More Matrices 899
11.17 Facts on Linear Equations and Least Squares 909
11.18 Notes 912
12.Functions, Limits, Sequences, Series, Infinite Products, and Derivatives 913
12.1 Open Sets and Closed Sets 913
12.2 Limits of Sequences 915
12.3 Series, Power Series, and Bi-power Series 919
12.4 Continuity 921
12.5 Derivatives 924
12.6 Complex-Valued Functions 926
12.7 Infinite Products 929
12.8 Functions of a Matrix 930
12.9 Matrix Square Root and Matrix Sign Functions 932
12.10 Vector and Matrix Derivatives 932
12.11 Facts on One Set 934
12.12 Facts on Two or More Sets 937
12.13 Facts on Functions 941
12.14 Facts on Functions of a Complex Variable 945
12.15 Facts on Functions of a Matrix 948
12.16 Facts on Derivatives 949
12.17 Facts on Limits of Functions 954
12.18 Facts on Limits of Sequences and Series 957
12.19 Notes 974
13.Infinite Series, Infinite Products, and Special Functions 975
13.1 Facts on Series for Subset, Eulerian, Partition, Bell, Ordered Bell, Bernoulli, Euler, and Up/Down Numbers 975
13.2 Facts on Bernoulli, Euler, Chebyshev, Legendre, Laguerre, Hermite, Bell, Ordered Bell, Harmonic, Fibonacci, and Lucas Polynomials 981
13.3 Facts on the Zeta, Gamma, Digamma, Generalized Harmonic, Dilogarithm, and Dirichlet L Functions 994
13.4 Facts on Power Series, Laurent Series, and Partial Fraction Expansions 1004
13.5 Facts on Series of Rational Functions 1021
13.6 Facts on Series of Trigonometric and Hyperbolic Functions 1057
13.7 Facts on Series of Binomial Coefficients 1063
13.8 Facts on Double-Summation Series 1071
13.9 Facts on Miscellaneous Series 1074
13.10 Facts on Infinite Products 1080
13.11 Notes 1092
14.Integrals 1093
14.1 Facts on Indefinite Integrals 1093
14.2 Facts on Definite Integrals of Rational Functions 1096
14.3 Facts on Definite Integrals of Radicals 1111
14.4 Facts on Definite Integrals of Trigonometric Functions 1114
14.5 Facts on Definite Integrals of Inverse Trigonometric Functions 1130
14.6 Facts on Definite Integrals of Logarithmic Functions 1132
14.7 Facts on Definite Integrals of Logarithmic, Trigonometric, and Hyperbolic Functions 1150
14.8 Facts on Definite Integrals of Exponential Functions 1157
14.9 Facts on Integral Representations of G and y 1169
14.10 Facts on Definite Integrals of the Gamma Function 1171
14.11 Facts on Integral Inequalities 1171
14.12 Facts on the Gaussian Density 1172
14.13 Facts on Multiple Integrals 1173
14.14 Notes 1178
15.The Matrix Exponential and Stability Theory 1179
15.1 Definition of the Matrix Exponential 1179
15.2 Structure of the Matrix Exponential 1181
15.3 Explicit Expressions 1185
15.4 Matrix Logarithms 1187
15.5 Principal Logarithm 1190
15.6 Lie Groups 1191
15.7 Linear Time-Varying Differential Equations 1193
15.8 Lyapunov Stability Theory 1195
15.9 Linear Stability Theory 1198
15.10 The Lyapunov Equation 1201
15.11 Discrete-Time Stability Theory 1203
15.12 Facts on Matrix Exponential Formulas 1204
15.13 Facts on the Matrix Sine and Cosine 1209
15.14 Facts on the Matrix Exponential for One Matrix 1209
15.15 Facts on the Matrix Exponential for Two or More Matrices 1211
15.16 Facts on the Matrix Exponential and Eigenvalues, Singular Values, and Norms for One Matrix 1217
15.17 Facts on the Matrix Exponential and Eigenvalues, Singular Values, and Norms for Two or More Matrices 1220
15.18 Facts on Stable Polynomials 1223
15.19 Facts on Stable Matrices 1226
15.20 Facts on Almost Nonnegative Matrices 1232
15.21 Facts on Discrete-Time-Stable Polynomials 1234
15.22 Facts on Discrete-Time-Stable Matrices 1239
15.23 Facts on Lie Groups 1243
15.24 Facts on Subspace Decomposition 1243
15.25 Notes 1247
16.Linear Systems and Control Theory 1249
16.1 State Space Models 1249
16.2 Laplace Transform Analysis and Transfer Functions 1252
16.3 The Unobservable Subspace and Observability 1253
16.4 Observable Asymptotic Stability 1257
16.5 Detectability 1259
16.6 The Controllable Subspace and Controllability 1259
16.7 Controllable Asymptotic Stability 1266
16.8 Stabilizability 1268
16.9 Realization Theory 1270
16.10 Zeros 1278
16.11 H2 System Norm 1285
16.12 Harmonic Steady-State Response 1288
16.13 System Interconnections 1289
16.14 Standard Control Problem 1291
16.15 Linear-Quadratic Control 1293
16.16 Solutions of the Riccati Equation 1295
16.17 The Stabilizing Solution of the Riccati Equation 1298
16.18 The Maximal Solution of the Riccati Equation 1302
16.19 Positive-Semidefinite and Positive-Definite Solutions of the Riccati Equation 1304
16.20 Facts on Linear Differential Equations 1305
16.21 Facts on Stability, Observability, and Controllability 1307
16.22 Facts on the Lyapunov Equation and Inertia 1309
16.23 Facts on the Discrete-Time Lyapunov Equation 1313
16.24 Facts on Realizations and the H2 System Norm 1313
16.25 Facts on the Riccati Equation 1316
16.26 Notes 1319
Bibliography 1321
Author Index 1433
Subject Index 1449