Model-Based Processing
An Applied Subspace Identification Approach
1st Edition
Published on 14. June 2019
Book
Hardback
544 pages
978-1-119-45776-3 (ISBN)
Description
A bridge between the application of subspace-based methods for parameter estimation in signal processing and subspace-based system identification in control systems
Model-Based Processing: An Applied Subspace Identification Approach provides expert insight on developing models for designing model-based signal processors (MBSP) employing subspace identification techniques to achieve model-based identification (MBID) and enables readers to evaluate overall performance using validation and statistical analysis methods. Focusing on subspace approaches to system identification problems, this book teaches readers to identify models quickly and incorporate them into various processing problems including state estimation, tracking, detection, classification, controls, communications, and other applications that require reliable models that can be adapted to dynamic environments.
The extraction of a model from data is vital to numerous applications, from the detection of submarines to determining the epicenter of an earthquake to controlling an autonomous vehicles-all requiring a fundamental understanding of their underlying processes and measurement instrumentation. Emphasizing real-world solutions to a variety of model development problems, this text demonstrates how model-based subspace identification system identification enables the extraction of a model from measured data sequences from simple time series polynomials to complex constructs of parametrically adaptive, nonlinear distributed systems. In addition, this resource features:
Kalman filtering for linear, linearized, and nonlinear systems; modern unscented Kalman filters; as well as Bayesian particle filters
Practical processor designs including comprehensive methods of performance analysis
Provides a link between model development and practical applications in model-based signal processing
Offers in-depth examination of the subspace approach that applies subspace algorithms to synthesized examples and actual applications
Enables readers to bridge the gap from statistical signal processing to subspace identification
Includes appendices, problem sets, case studies, examples, and notes for MATLAB
Model-Based Processing: An Applied Subspace Identification Approach is essential reading for advanced undergraduate and graduate students of engineering and science as well as engineers working in industry and academia.
Model-Based Processing: An Applied Subspace Identification Approach provides expert insight on developing models for designing model-based signal processors (MBSP) employing subspace identification techniques to achieve model-based identification (MBID) and enables readers to evaluate overall performance using validation and statistical analysis methods. Focusing on subspace approaches to system identification problems, this book teaches readers to identify models quickly and incorporate them into various processing problems including state estimation, tracking, detection, classification, controls, communications, and other applications that require reliable models that can be adapted to dynamic environments.
The extraction of a model from data is vital to numerous applications, from the detection of submarines to determining the epicenter of an earthquake to controlling an autonomous vehicles-all requiring a fundamental understanding of their underlying processes and measurement instrumentation. Emphasizing real-world solutions to a variety of model development problems, this text demonstrates how model-based subspace identification system identification enables the extraction of a model from measured data sequences from simple time series polynomials to complex constructs of parametrically adaptive, nonlinear distributed systems. In addition, this resource features:
Kalman filtering for linear, linearized, and nonlinear systems; modern unscented Kalman filters; as well as Bayesian particle filters
Practical processor designs including comprehensive methods of performance analysis
Provides a link between model development and practical applications in model-based signal processing
Offers in-depth examination of the subspace approach that applies subspace algorithms to synthesized examples and actual applications
Enables readers to bridge the gap from statistical signal processing to subspace identification
Includes appendices, problem sets, case studies, examples, and notes for MATLAB
Model-Based Processing: An Applied Subspace Identification Approach is essential reading for advanced undergraduate and graduate students of engineering and science as well as engineers working in industry and academia.
More details
Language
English
Place of publication
New York
United States
Target group
Professional and scholarly
Product notice
sewn/stitched
Cloth over boards
Dimensions
Height: 231 mm
Width: 157 mm
Thickness: 30 mm
Weight
998 gr
ISBN-13
978-1-119-45776-3 (9781119457763)
Copyright in bibliographic data and cover images is held by Nielsen Book Services Limited or by the publishers or by their respective licensors: all rights reserved.
Schweitzer Classification
Other editions
Additional editions
E-Book
03/2019
1st Edition
Wiley
€125.99
Available for download
E-Book
03/2019
1st Edition
Wiley
€125.99
Available for download
Person
JAMES V. CANDY, PHD, is Chief Scientist for Engineering, Distinguished Member of the Technical Staff, and founder of the Center for Advanced Signal & Image Sciences (CASIS), Lawrence Livermore National Laboratory, Livermore, California. Dr. Candy is also Adjunct Full-Professor, University of California, Santa Barbara, a Fellow of the IEEE, and a Fellow of the Acoustical Society of America. He is author of Bayesian Signal Processing: Classical, Modern, and Particle Filtering Methods and Model-Based Signal Processing (John Wiley & Sons, Inc., 2006) and Bayesian Signal Processing: Classical, Modern and Particle Filtering Methods, Second Edition (John Wiley & Sons, Inc., 2016). Dr. Candy was awarded the IEEE Distinguished Technical Achievement Award for his development of model-based signal processing and the Acoustical Society of America Helmholtz-Rayleigh Interdisciplinary Silver Medal for his contributions to acoustical signal processing and underwater acoustics.
Content
Preface xiii
Acknowledgements xxi
Glossary xxiii
1 Introduction 1
1.1 Background 1
1.2 Signal Estimation 2
1.3 Model-Based Processing 8
1.4 Model-Based Identification 16
1.5 Subspace Identification 20
1.6 Notation and Terminology 22
1.7 Summary 24
MATLAB Notes 25
References 25
Problems 26
2 Random Signals and Systems 29
2.1 Introduction 29
2.2 Discrete Random Signals 32
2.3 Spectral Representation of Random Signals 36
2.4 Discrete Systems with Random Inputs 40
2.4.1 Spectral Theorems 41
2.4.2 ARMAX Modeling 42
2.5 Spectral Estimation 44
2.5.1 Classical (Nonparametric) Spectral Estimation 44
2.5.1.1 Correlation Method (Blackman-Tukey) 45
2.5.1.2 Average Periodogram Method (Welch) 46
2.5.2 Modern (Parametric) Spectral Estimation 47
2.5.2.1 Autoregressive (All-Pole) Spectral Estimation 48
2.5.2.2 Autoregressive Moving Average Spectral Estimation 51
2.5.2.3 Minimum Variance Distortionless Response (MVDR) Spectral Estimation 52
2.5.2.4 Multiple Signal Classification (MUSIC) Spectral Estimation 55
2.6 Case Study: Spectral Estimation of Bandpass Sinusoids 59
2.7 Summary 61
MATLAB Notes 61
References 62
Problems 64
3 State-Space Models for Identification 69
3.1 Introduction 69
3.2 Continuous-Time State-Space Models 69
3.3 Sampled-Data State-Space Models 73
3.4 Discrete-Time State-Space Models 74
3.4.1 Linear Discrete Time-Invariant Systems 77
3.4.2 Discrete Systems Theory 78
3.4.3 Equivalent Linear Systems 82
3.4.4 Stable Linear Systems 83
3.5 Gauss-Markov State-Space Models 83
3.5.1 Discrete-Time Gauss-Markov Models 83
3.6 Innovations Model 89
3.7 State-Space Model Structures 90
3.7.1 Time-Series Models 91
3.7.2 State-Space and Time-Series Equivalence Models 91
3.8 Nonlinear (Approximate) Gauss-Markov State-Space Models 97
3.9 Summary 101
MATLAB Notes 102
References 102
Problems 103
4 Model-Based Processors 107
4.1 Introduction 107
4.2 Linear Model-Based Processor: Kalman Filter 108
4.2.1 Innovations Approach 110
4.2.2 Bayesian Approach 114
4.2.3 Innovations Sequence 116
4.2.4 Practical Linear Kalman Filter Design: Performance Analysis 117
4.2.5 Steady-State Kalman Filter 125
4.2.6 Kalman Filter/Wiener Filter Equivalence 128
4.3 Nonlinear State-Space Model-Based Processors 129
4.3.1 Nonlinear Model-Based Processor: Linearized Kalman Filter 130
4.3.2 Nonlinear Model-Based Processor: Extended Kalman Filter 133
4.3.3 Nonlinear Model-Based Processor: Iterated-Extended Kalman Filter 138
4.3.4 Nonlinear Model-Based Processor: Unscented Kalman Filter 141
4.3.5 Practical Nonlinear Model-Based Processor Design: Performance Analysis 148
4.3.6 Nonlinear Model-Based Processor: Particle Filter 151
4.3.7 Practical Bayesian Model-Based Design: Performance Analysis 160
4.4 Case Study: 2D-Tracking Problem 166
4.5 Summary 173
MATLAB Notes 173
References 174
Problems 177
5 Parametrically Adaptive Processors 185
5.1 Introduction 185
5.2 Parametrically Adaptive Processors: Bayesian Approach 186
5.3 Parametrically Adaptive Processors: Nonlinear Kalman Filters 187
5.3.1 Parametric Models 188
5.3.2 Classical Joint State/Parametric Processors: Augmented Extended Kalman Filter 190
5.3.3 Modern Joint State/Parametric Processor: Augmented Unscented Kalman Filter 198
5.4 Parametrically Adaptive Processors: Particle Filter 201
5.4.1 Joint State/Parameter Estimation: Particle Filter 201
5.5 Parametrically Adaptive Processors: Linear Kalman Filter 208
5.6 Case Study: Random Target Tracking 214
5.7 Summary 222
MATLAB Notes 223
References 223
Problems 226
6 Deterministic Subspace Identification 231
6.1 Introduction 231
6.2 Deterministic Realization Problem 232
6.2.1 Realization Theory 233
6.2.2 Balanced Realizations 238
6.2.3 Systems Theory Summary 239
6.3 Classical Realization 241
6.3.1 Ho-Kalman Realization Algorithm 241
6.3.2 SVD Realization Algorithm 243
6.3.2.1 Realization: Linear Time-Invariant Mechanical Systems 246
6.3.3 Canonical Realization 251
6.3.3.1 Invariant System Descriptions 251
6.3.3.2 Canonical Realization Algorithm 257
6.4 Deterministic Subspace Realization: Orthogonal Projections 264
6.4.1 Subspace Realization: Orthogonal Projections 266
6.4.2 Multivariable Output Error State-Space (MOESP) Algorithm 271
6.5 Deterministic Subspace Realization: Oblique Projections 274
6.5.1 Subspace Realization: Oblique Projections 278
6.5.2 Numerical Algorithms for Subspace State-Space System Identification (N4SID) Algorithm 280
6.6 Model Order Estimation and Validation 285
6.6.1 Order Estimation: SVD Approach 286
6.6.2 Model Validation 289
6.7 Case Study: Structural Vibration Response 295
6.8 Summary 299
MATLAB Notes 300
References 300
Problems 303
7 Stochastic Subspace Identification 309
7.1 Introduction 309
7.2 Stochastic Realization Problem 312
7.2.1 Correlated Gauss-Markov Model 312
7.2.2 Gauss-Markov Power Spectrum 313
7.2.3 Gauss-Markov Measurement Covariance 314
7.2.4 Stochastic Realization Theory 315
7.3 Classical Stochastic Realization via the Riccati Equation 317
7.4 Classical Stochastic Realization via Kalman Filter 321
7.4.1 Innovations Model 321
7.4.2 Innovations Power Spectrum 322
7.4.3 Innovations Measurement Covariance 323
7.4.4 Stochastic Realization: Innovations Model 325
7.5 Stochastic Subspace Realization: Orthogonal Projections 330
7.5.1 Multivariable Output Error State-SPace (MOESP) Algorithm 334
7.6 Stochastic Subspace Realization: Oblique Projections 342
7.6.1 Numerical Algorithms for Subspace State-Space System Identification (N4SID) Algorithm 346
7.6.2 Relationship: Oblique (N4SID) and Orthogonal (MOESP) Algorithms 351
7.7 Model Order Estimation and Validation 353
7.7.1 Order Estimation: Stochastic Realization Problem 354
7.7.1.1 Order Estimation: Statistical Methods 356
7.7.2 Model Validation 362
7.7.2.1 Residual Testing 363
7.8 Case Study: Vibration Response of a Cylinder: Identification and Tracking 369
7.9 Summary 378
MATLAB NOTES 378
References 379
Problems 382
8 Subspace Processors for Physics-Based Application 391
8.1 Subspace Identification of a Structural Device 391
8.1.1 State-Space Vibrational Systems 392
8.1.1.1 State-Space Realization 394
8.1.2 Deterministic State-Space Realizations 396
8.1.2.1 Subspace Approach 396
8.1.3 Vibrational System Processing 398
8.1.4 Application: Vibrating Structural Device 400
8.1.5 Summary 404
8.2 MBID for Scintillator System Characterization 405
8.2.1 Scintillation Pulse Shape Model 407
8.2.2 Scintillator State-Space Model 409
8.2.3 Scintillator Sampled-Data State-Space Model 410
8.2.4 Gauss-Markov State-Space Model 411
8.2.5 Identification of the Scintillator Pulse Shape Model 412
8.2.6 Kalman Filter Design: Scintillation/Photomultiplier System 414
8.2.6.1 Kalman Filter Design: Scintillation/Photomultiplier Data 416
8.2.7 Summary 417
8.3 Parametrically Adaptive Detection of Fission Processes 418
8.3.1 Fission-Based Processing Model 419
8.3.2 Interarrival Distribution 420
8.3.3 Sequential Detection 422
8.3.4 Sequential Processor 422
8.3.5 Sequential Detection for Fission Processes 424
8.3.6 Bayesian Parameter Estimation 426
8.3.7 Sequential Bayesian Processor 427
8.3.8 Particle Filter for Fission Processes 429
8.3.9 SNM Detection and Estimation: Synthesized Data 430
8.3.10 Summary 433
8.4 Parametrically Adaptive Processing for Shallow Ocean Application 435
8.4.1 State-Space Propagator 436
8.4.2 State-Space Model 436
8.4.2.1 Augmented State-Space Models 438
8.4.3 Processors 441
8.4.4 Model-Based Ocean Acoustic Processing 444
8.4.4.1 Adaptive PF Design: Modal Coefficients 445
8.4.4.2 Adaptive PF Design: Wavenumbers 447
8.4.5 Summary 450
8.5 MBID for Chirp Signal Extraction 452
8.5.1 Chirp-like Signals 453
8.5.1.1 Linear Chirp 453
8.5.1.2 Frequency-Shift Key (FSK) Signal 455
8.5.2 Model-Based Identification: Linear Chirp Signals 457
8.5.2.1 Gauss-Markov State-Space Model: Linear Chirp 457
8.5.3 Model-Based Identification: FSK Signals 459
8.5.3.1 Gauss-Markov State-Space Model: FSK Signals 460
8.5.4 Summary 462
References 462
Appendix A Probability and Statistics Overview 467
A.1 Probability Theory 467
A.2 Gaussian Random Vectors 473
A.3 Uncorrelated Transformation: Gaussian Random Vectors 473
A.4 Toeplitz Correlation Matrices 474
A.5 Important Processes 474
References 476
Appendix B Projection Theory 477
B.1 Projections: Deterministic Spaces 477
B.2 Projections: Random Spaces 478
B.3 Projection: Operators 479
B.3.1 Orthogonal (Perpendicular) Projections 479
B.3.2 Oblique (Parallel) Projections 481
References 483
Appendix C Matrix Decompositions 485
C.1 Singular-Value Decomposition 485
C.2 QR-Decomposition 487
C.3 LQ-Decomposition 487
References 488
Appendix D Output-Only Subspace Identification 489
References 492
Index 495
Acknowledgements xxi
Glossary xxiii
1 Introduction 1
1.1 Background 1
1.2 Signal Estimation 2
1.3 Model-Based Processing 8
1.4 Model-Based Identification 16
1.5 Subspace Identification 20
1.6 Notation and Terminology 22
1.7 Summary 24
MATLAB Notes 25
References 25
Problems 26
2 Random Signals and Systems 29
2.1 Introduction 29
2.2 Discrete Random Signals 32
2.3 Spectral Representation of Random Signals 36
2.4 Discrete Systems with Random Inputs 40
2.4.1 Spectral Theorems 41
2.4.2 ARMAX Modeling 42
2.5 Spectral Estimation 44
2.5.1 Classical (Nonparametric) Spectral Estimation 44
2.5.1.1 Correlation Method (Blackman-Tukey) 45
2.5.1.2 Average Periodogram Method (Welch) 46
2.5.2 Modern (Parametric) Spectral Estimation 47
2.5.2.1 Autoregressive (All-Pole) Spectral Estimation 48
2.5.2.2 Autoregressive Moving Average Spectral Estimation 51
2.5.2.3 Minimum Variance Distortionless Response (MVDR) Spectral Estimation 52
2.5.2.4 Multiple Signal Classification (MUSIC) Spectral Estimation 55
2.6 Case Study: Spectral Estimation of Bandpass Sinusoids 59
2.7 Summary 61
MATLAB Notes 61
References 62
Problems 64
3 State-Space Models for Identification 69
3.1 Introduction 69
3.2 Continuous-Time State-Space Models 69
3.3 Sampled-Data State-Space Models 73
3.4 Discrete-Time State-Space Models 74
3.4.1 Linear Discrete Time-Invariant Systems 77
3.4.2 Discrete Systems Theory 78
3.4.3 Equivalent Linear Systems 82
3.4.4 Stable Linear Systems 83
3.5 Gauss-Markov State-Space Models 83
3.5.1 Discrete-Time Gauss-Markov Models 83
3.6 Innovations Model 89
3.7 State-Space Model Structures 90
3.7.1 Time-Series Models 91
3.7.2 State-Space and Time-Series Equivalence Models 91
3.8 Nonlinear (Approximate) Gauss-Markov State-Space Models 97
3.9 Summary 101
MATLAB Notes 102
References 102
Problems 103
4 Model-Based Processors 107
4.1 Introduction 107
4.2 Linear Model-Based Processor: Kalman Filter 108
4.2.1 Innovations Approach 110
4.2.2 Bayesian Approach 114
4.2.3 Innovations Sequence 116
4.2.4 Practical Linear Kalman Filter Design: Performance Analysis 117
4.2.5 Steady-State Kalman Filter 125
4.2.6 Kalman Filter/Wiener Filter Equivalence 128
4.3 Nonlinear State-Space Model-Based Processors 129
4.3.1 Nonlinear Model-Based Processor: Linearized Kalman Filter 130
4.3.2 Nonlinear Model-Based Processor: Extended Kalman Filter 133
4.3.3 Nonlinear Model-Based Processor: Iterated-Extended Kalman Filter 138
4.3.4 Nonlinear Model-Based Processor: Unscented Kalman Filter 141
4.3.5 Practical Nonlinear Model-Based Processor Design: Performance Analysis 148
4.3.6 Nonlinear Model-Based Processor: Particle Filter 151
4.3.7 Practical Bayesian Model-Based Design: Performance Analysis 160
4.4 Case Study: 2D-Tracking Problem 166
4.5 Summary 173
MATLAB Notes 173
References 174
Problems 177
5 Parametrically Adaptive Processors 185
5.1 Introduction 185
5.2 Parametrically Adaptive Processors: Bayesian Approach 186
5.3 Parametrically Adaptive Processors: Nonlinear Kalman Filters 187
5.3.1 Parametric Models 188
5.3.2 Classical Joint State/Parametric Processors: Augmented Extended Kalman Filter 190
5.3.3 Modern Joint State/Parametric Processor: Augmented Unscented Kalman Filter 198
5.4 Parametrically Adaptive Processors: Particle Filter 201
5.4.1 Joint State/Parameter Estimation: Particle Filter 201
5.5 Parametrically Adaptive Processors: Linear Kalman Filter 208
5.6 Case Study: Random Target Tracking 214
5.7 Summary 222
MATLAB Notes 223
References 223
Problems 226
6 Deterministic Subspace Identification 231
6.1 Introduction 231
6.2 Deterministic Realization Problem 232
6.2.1 Realization Theory 233
6.2.2 Balanced Realizations 238
6.2.3 Systems Theory Summary 239
6.3 Classical Realization 241
6.3.1 Ho-Kalman Realization Algorithm 241
6.3.2 SVD Realization Algorithm 243
6.3.2.1 Realization: Linear Time-Invariant Mechanical Systems 246
6.3.3 Canonical Realization 251
6.3.3.1 Invariant System Descriptions 251
6.3.3.2 Canonical Realization Algorithm 257
6.4 Deterministic Subspace Realization: Orthogonal Projections 264
6.4.1 Subspace Realization: Orthogonal Projections 266
6.4.2 Multivariable Output Error State-Space (MOESP) Algorithm 271
6.5 Deterministic Subspace Realization: Oblique Projections 274
6.5.1 Subspace Realization: Oblique Projections 278
6.5.2 Numerical Algorithms for Subspace State-Space System Identification (N4SID) Algorithm 280
6.6 Model Order Estimation and Validation 285
6.6.1 Order Estimation: SVD Approach 286
6.6.2 Model Validation 289
6.7 Case Study: Structural Vibration Response 295
6.8 Summary 299
MATLAB Notes 300
References 300
Problems 303
7 Stochastic Subspace Identification 309
7.1 Introduction 309
7.2 Stochastic Realization Problem 312
7.2.1 Correlated Gauss-Markov Model 312
7.2.2 Gauss-Markov Power Spectrum 313
7.2.3 Gauss-Markov Measurement Covariance 314
7.2.4 Stochastic Realization Theory 315
7.3 Classical Stochastic Realization via the Riccati Equation 317
7.4 Classical Stochastic Realization via Kalman Filter 321
7.4.1 Innovations Model 321
7.4.2 Innovations Power Spectrum 322
7.4.3 Innovations Measurement Covariance 323
7.4.4 Stochastic Realization: Innovations Model 325
7.5 Stochastic Subspace Realization: Orthogonal Projections 330
7.5.1 Multivariable Output Error State-SPace (MOESP) Algorithm 334
7.6 Stochastic Subspace Realization: Oblique Projections 342
7.6.1 Numerical Algorithms for Subspace State-Space System Identification (N4SID) Algorithm 346
7.6.2 Relationship: Oblique (N4SID) and Orthogonal (MOESP) Algorithms 351
7.7 Model Order Estimation and Validation 353
7.7.1 Order Estimation: Stochastic Realization Problem 354
7.7.1.1 Order Estimation: Statistical Methods 356
7.7.2 Model Validation 362
7.7.2.1 Residual Testing 363
7.8 Case Study: Vibration Response of a Cylinder: Identification and Tracking 369
7.9 Summary 378
MATLAB NOTES 378
References 379
Problems 382
8 Subspace Processors for Physics-Based Application 391
8.1 Subspace Identification of a Structural Device 391
8.1.1 State-Space Vibrational Systems 392
8.1.1.1 State-Space Realization 394
8.1.2 Deterministic State-Space Realizations 396
8.1.2.1 Subspace Approach 396
8.1.3 Vibrational System Processing 398
8.1.4 Application: Vibrating Structural Device 400
8.1.5 Summary 404
8.2 MBID for Scintillator System Characterization 405
8.2.1 Scintillation Pulse Shape Model 407
8.2.2 Scintillator State-Space Model 409
8.2.3 Scintillator Sampled-Data State-Space Model 410
8.2.4 Gauss-Markov State-Space Model 411
8.2.5 Identification of the Scintillator Pulse Shape Model 412
8.2.6 Kalman Filter Design: Scintillation/Photomultiplier System 414
8.2.6.1 Kalman Filter Design: Scintillation/Photomultiplier Data 416
8.2.7 Summary 417
8.3 Parametrically Adaptive Detection of Fission Processes 418
8.3.1 Fission-Based Processing Model 419
8.3.2 Interarrival Distribution 420
8.3.3 Sequential Detection 422
8.3.4 Sequential Processor 422
8.3.5 Sequential Detection for Fission Processes 424
8.3.6 Bayesian Parameter Estimation 426
8.3.7 Sequential Bayesian Processor 427
8.3.8 Particle Filter for Fission Processes 429
8.3.9 SNM Detection and Estimation: Synthesized Data 430
8.3.10 Summary 433
8.4 Parametrically Adaptive Processing for Shallow Ocean Application 435
8.4.1 State-Space Propagator 436
8.4.2 State-Space Model 436
8.4.2.1 Augmented State-Space Models 438
8.4.3 Processors 441
8.4.4 Model-Based Ocean Acoustic Processing 444
8.4.4.1 Adaptive PF Design: Modal Coefficients 445
8.4.4.2 Adaptive PF Design: Wavenumbers 447
8.4.5 Summary 450
8.5 MBID for Chirp Signal Extraction 452
8.5.1 Chirp-like Signals 453
8.5.1.1 Linear Chirp 453
8.5.1.2 Frequency-Shift Key (FSK) Signal 455
8.5.2 Model-Based Identification: Linear Chirp Signals 457
8.5.2.1 Gauss-Markov State-Space Model: Linear Chirp 457
8.5.3 Model-Based Identification: FSK Signals 459
8.5.3.1 Gauss-Markov State-Space Model: FSK Signals 460
8.5.4 Summary 462
References 462
Appendix A Probability and Statistics Overview 467
A.1 Probability Theory 467
A.2 Gaussian Random Vectors 473
A.3 Uncorrelated Transformation: Gaussian Random Vectors 473
A.4 Toeplitz Correlation Matrices 474
A.5 Important Processes 474
References 476
Appendix B Projection Theory 477
B.1 Projections: Deterministic Spaces 477
B.2 Projections: Random Spaces 478
B.3 Projection: Operators 479
B.3.1 Orthogonal (Perpendicular) Projections 479
B.3.2 Oblique (Parallel) Projections 481
References 483
Appendix C Matrix Decompositions 485
C.1 Singular-Value Decomposition 485
C.2 QR-Decomposition 487
C.3 LQ-Decomposition 487
References 488
Appendix D Output-Only Subspace Identification 489
References 492
Index 495