Optimal and Robust State Estimation
Description
Optimal and Robust State Estimation: Finite Impulse Response (FIR) and Kalman Approaches is a comprehensive investigation into batch state estimators and recursive forms. The work begins by introducing the reader to the state estimation approach and provides a brief historical overview. Next, the work discusses the specific properties of finite impulse response (FIR) state estimators. Further chapters give the basics of probability and stochastic processes, discuss the available linear and nonlinear state estimators, deal with optimal FIR filtering, and consider a limited memory batch and recursive algorithms.
Other topics covered include solving the q-lag FIR smoothing problem, introducing the receding horizon (RH) FIR state estimation approach, and developing the theory of FIR state estimation under disturbances. The book closes by discussing the theory of FIR state estimation for uncertain systems and providing several applications where the FIR state estimators are used effectively. Key concepts covered in the work include:
* A holistic overview of the state estimation approach, which arose from the need to know the internal state of a real system, given that the input and output are both known
* Optimal, optimal unbiased, maximum likelihood, and unbiased and robust finite impulse response (FIR) structures
* FIR state estimation approach along with the infinite impulse response (IIR) and Kalman approaches
* Cost functions and the most critical properties of FIR and IIR state estimates
Optimal and Robust State Estimation: Finite Impulse Response (FIR) and Kalman Approaches was written for professionals in the fields of microwave engineering, system engineering, and robotics who wish to move towards solving finite impulse response (FIR) estimate issues in both theoretical and practical applications. Graduate and senior undergraduate students with coursework dealing with state estimation will also be able to use the book to gain a valuable foundation of knowledge and become more adept in their chosen fields of study.
More details
Other editions
Additional editions
Persons
YURIY S. SHMALIY, PhD, is a Professor with the Universidad de Guanajuato, Mexico. He serves as an Editorial Board Member in various scientific journals and is an IEEE Fellow. He also developed the theory of FIR state estimation, gave many keynote and plenary lectures, and his discrete orthogonal polynomials are called discrete Shmaliy moments.
SHUNYI ZHAO, PhD, is a Professor with the Jiangnan University, China. His current research interests include statistical signal processing, Bayesian estimation theory, and fault detection and diagnosis.
Content
1 Introduction 1
1.1 What is System State? 2
1.1.1 Why and How do We Estimate State? 2
1.1.2 What Model to Estimate State? 3
1.1.3 What are Basic State Estimates in Discrete Time? 5
1.2 Properties of State Estimators 6
1.2.1 Structures and Types 6
1.2.2 Optimality 10
1.2.3 Unbiased Optimality (Maximum Likelihood) 11
1.2.4 Suboptimality 14
1.2.5 Unbiasedness 17
1.2.6 Deadbeat 17
1.2.7 Denoising (Noise Power Gain) 17
1.2.8 Stability 18
1.2.9 Robustness 18
1.2.10 Computational Complexity 19
1.2.11 Memory Use 20
1.3 More About FIR State Estimators 20
1.4 Historical Overview and Most Noticeable Works 21
1.5 Summary 26
1.6 Problems 27
2 Probability and Stochastic Processes 31
2.1 Random Variables 31
2.1.1 Moments and Cumulants 33
2.1.2 Product Moments 39
2.1.3 Vector Random Variables 41
2.1.4 Conditional Probability. Bayes' Rule 42
2.1.5 Transformation of Random Variables 45
2.2 Stochastic Processes 47
2.2.1 Correlation Function 48
2.2.2 Power Spectral Density 51
2.2.3 Gaussian Processes 53
2.2.4 White Gaussian Noise 55
2.2.5 Markov Processes 57
2.3 Stochastic Differential Equation 60
2.3.1 Standard Stochastic Differential Equation 61
2.3.2 It^o and Stratonovich Stochastic Calculus 61
2.3.3 Diffusion Process Interpretation 62
2.3.4 Fokker-Planck-Kolmogorov Equation 63
2.3.5 Langevin Equation 64
2.4 Summary 65
2.5 Problems 66
3 State Estimation 71
3.1 Lineal Stochastic Process in State Space 71
3.1.1 Continuous-Time Model 73
3.1.2 Discrete-Time Model 77
3.2 Methods of Linear State Estimation 81
3.2.1 Bayesian Estimator 82
3.2.2 Maximum Likelihood Estimator 85
3.2.3 Least Squares Estimator 86
3.2.4 Unbiased Estimator 87
3.2.5 Kalman Filtering Algorithm 88
3.2.6 Backward Kalman Filter 94
3.2.7 Alternative Forms of Kalman Filter 96
3.2.8 General Kalman Filter 98
3.2.9 Kalman-Bucy Filter 110
3.3 Linear Recursive Smoothing 113
3.3.1 Rauch-Tung-Striebel Algorithm 113
3.3.2 Bryson-Frazier Algorithm 114
3.3.3 Two-Filter (Forward-Backward) Smoothing 115
3.4 Nonlinear Models and Estimators 116
3.4.1 Extended Kalman Filter 117
3.4.2 Unscented Kalman Filter 119
3.4.3 Particle Filtering 122
3.5 Robust State Estimation 126
3.5.1 Robustified Kalman Filter 127
3.5.2 Robust Kalman Filter 128
3.5.3 H8 Filtering 131
3.5.4 Game Theory H8 Filter 132
3.6 Summary 133
3.7 Problems 134
4 Optimal FIR and Limited Memory Filtering 139
4.1 Extended State-Space Model 140
4.2 The a posteriori Optimal FIR Filter 142
4.2.1 Batch Estimate and Error Covariance 143
4.2.2 Recursive Forms 145
4.2.3 System Identification 149
4.3 The a posteriori Optimal Unbiased FIR Filter 149
4.3.1 Batch OUFIR-I Estimate and Error Covariance 150
4.3.2 Recursive Forms for OUFIR-I Filter 151
4.3.3 Batch OUFIR-II Estimate and Error Covariance 153
4.3.4 Recursion Forms for OUFIR-II Filter 154
4.4 Maximum Likelihood FIR Estimator 158
4.4.1 ML-I FIR Filtering Estimate 158
4.4.2 Equivalence of ML-I FIR and OUFIR Filters 159
4.4.3 ML-II FIR Filtering Estimate 162
4.4.4 Properties of ML FIR State Estimators 163
4.5 The a priori FIR Filters 164
4.5.1 The a priori Optimal FIR Filter 164
4.5.2 The a priori Optimal Unbiased FIR Filter 165
4.6 Limited Memory Filtering 165
4.6.1 Batch Limited Memory Filter 166
4.6.2 Iterative LMF Algorithm using Recursions 168
4.7 Continuous-Time Optimal FIR Filter 169
4.7.1 Optimal Impulse Response 169
4.7.2 Differential Equation Form 171
4.8 Extended a posteriori OFIR Filtering 172
4.9 Properties of FIR State Estimators 174
4.10 Summary 179
4.11 Problems 182
5 Optimal FIR Smoothing 187
5.1 Introduction 187
5.2 Smoothing Problem 188
5.3 Forward Filter/Forward Model q-lag OFIR Smoothing 189
5.3.1 Batch Smoothing Estimate 190
5.3.2 Error Covariance 193
5.4 Backward OFIR Filtering 195
5.4.1 Backward State-Space Model 195
5.4.2 Batch Estimate 196
5.4.3 Recursive Estimate and Error Covariance 198
5.5 Backward Filter/Backward Model g-lag OFIR Smoother 202
5.5.1 Batch Smoothing Estimate 203
5.5.2 Error Covariance 204
5.6 Forward Filter/Backward Model q-Lag OFIR Smoother 205
5.6.1 Batch Smoothing Estimate 205
5.6.2 Error Covariance 208
5.7 Backward Filter/Forward Model q-Lag OFIR Smoother 208
5.7.1 Batch Smoothing Estimate 208
5.7.2 Error Covariance 211
5.8 Two-Filter q-lag OFIR Smoother 213
5.9 q-Lag ML FIR Smoothing 214
5.9.1 Batch q-lag ML FIR Estimate 215
5.9.2 Error Covariance 216
5.10 Summary 216
5.11 Problems 217
6 Unbiased FIR State Estimation 221
6.1 Introduction 221
6.2 The a posteriori UFIR Filter 222
6.2.1 Batch Form 222
6.2.2 Iterative Algorithm Using Recursions 224
6.2.3 Recursive Error Covariance 226
6.2.4 Optimal Averaging Horizon 228
6.3 Backward a posteriori UFIR Filter 234
6.3.1 Batch Form 235
6.3.2 Recursions and Iterative Algorithm 236
6.3.3 Recursive Error Covariance 239
6.4 The q-lag UFIR Smoother 240
6.4.1 Batch and Recursive Forms 240
6.4.2 Error Covariance 242
6.4.3 Equivalence of UFIR Smoothers 244
6.5 State Estimation using Polynomial Models 245
6.5.1 Problems Solved with UFIR Structures 246
6.5.2 The p-shift UFIR Filter 247
6.5.3 Filtering of Polynomial Models 250
6.5.4 Discrete Shmaliy Moments 252
6.5.5 Smoothing Filtering and Smoothing 252
6.5.6 Generalized Savitzky-Golay Filter 254
6.5.7 Predictive Filtering and Prediction 255
6.6 UFIR State Estimation under Colored Noise 256
6.6.1 Colored Measurement Noise 256
6.6.2 Colored Process Noise 259
6.7 Extended UFIR Filtering 262
6.7.1 First-Order Extended UFIR Filter 263
6.7.2 Second-Order Extended UFIR Filter 263
6.8 Robustness of UFIR Filter 266
6.8.1 Errors in Noise Covariances and Weighted Matrices 268
6.8.2 Model Errors 271
6.8.3 Temporary Uncertainties 274
6.9 Implementation of Polynomial UFIR Filters 276
6.9.1 Filter Structures in z-Domain 276
6.9.2 Transfer Function in DFT Domain 282
6.10 Summary 287
6.11 Problems 288
7 FIR Prediction and Receding Horizon Filtering 295
7.1 Introduction 295
7.2 Prediction Strategies 296
7.2.1 Kalman Predictor 296
7.3 Extended Predictive State-Space Model 298
7.4 UFIR Predictor 298
7.4.1 Batch UFIR Predictor 299
7.4.2 Iterative Algorithm using Recursions 299
7.4.3 Recursive Error Covariance 303
7.5 Optimal FIR Predictor 304
7.5.1 Batch Estimate and Error Covariance 305
7.5.2 Recursive Forms and Iterative Algorithm 306
7.6 Receding Horizon FIR Filtering 308
7.6.1 MVF-I Filter for Stationary Processes 309
7.6.2 MVF-II Filter for Nonstationary Processes 311
7.7 Maximum Likelihood FIR Predictor 313
7.7.1 ML-I FIR Predictor 314
7.7.2 ML-II FIR Predictor 315
7.8 Extended OFIR Prediction 315
7.9 Summary 317
7.10 Problems 318
8 Robust FIR State Estimation under Disturbances 323
8.1 Extended Models under Disturbances 324
8.2 The a posteriori H2 FIR Filtering 326
8.2.1 H2-OFIR Filter 328
8.2.2 Optimal Unbiased H2 FIR Filter 330
8.2.3 Suboptimal H2 FIR Filtering Algorithms 336
8.3 H2 FIR Prediction 338
8.3.1 H2-OFIR Predictor 339
8.3.2 Bias-constrained H2-OUFIR Predictor 341
8.3.3 Suboptimal H2 FIR Predictive Algorithms 341
8.3.4 Receding Horizon H2-MVF Filter 343
8.4 H8 FIR State Estimation 344
8.4.1 The a posteriori H8 FIR Filter 346
8.4.2 H8 FIR Predictor 350
8.5 H2{H8 FIR Filter and Predictor 354
8.6 Generalized H2 FIR State Estimation 355
8.6.1 Energy-to-Peak Lemma 355
8.6.2 L2-to-L8 FIR Filter and Predictor 359
8.7 L1 FIR State Estimation 362
8.7.1 Peak-to-Peak Lemma 363
8.7.2 L8-to-L8 FIR Filtering and Prediction 365
8.8 Game Theory FIR State Estimation 367
8.8.1 The a posteriori Energy-to-Power FIR Filter 368
8.8.2 Energy-to-Power FIR Predictor 370
8.9 Recursive Computation of Robust FIR Estimates 371
8.9.1 Uncontrolled Processes 372
8.9.2 Controlled Processes 372
8.10 FIR Smoothing under Disturbances 374
8.11 Summary 374
8.12 Problems 376
9 Robust FIR State Estimation for Uncertain Systems 379
9.1 Extended Models for Uncertain Systems 380
9.2 The a posteriori H2 FIR Filtering 386
9.2.1 H2-OFIR Filter 387
9.2.2 Bias-constrained H2-OFIR Filter 392
9.3 H2 FIR Prediction 394
9.3.1 Optimal H2 FIR Predictor 395
9.3.2 Bias-constrained H2-OUFIR Predictor 399
9.4 Suboptimal H2 FIR Structures using LMI 400
9.4.1 Suboptimal H2 FIR Filter 401
9.4.2 Bias-Constrained Suboptimal H2 FIR Filter 402
9.4.3 Suboptimal H2 FIR Predictor 403
9.4.4 Bias-Constrained Suboptimal H2 FIR Predictor 404
9.5 H8 FIR State Estimation for Uncertain Systems 405
9.5.1 The a posteriori H8 FIR Filter 405
9.5.2 H8 FIR Predictor 407
9.6 Hybrid H2{H8 FIR Structures 410
9.7 Generalized H2 FIR Structures for Uncertain Systems 411
9.7.1 The a posteriori L2-to-L8 FIR Filter 412
9.7.2 L2-to-L8 FIR Predictor 414
9.8 Robust L1 FIR Structures for Uncertain Systems 416
9.8.1 The a posteriori L8-to-L8 FIR Filter 417
9.8.2 L8-to-L8 FIR Predictor 417
9.9 Summary 418
9.10 Problems 419
10 Advanced Topics in FIR State Estimation 423
10.1 Distributed Filtering over Networks 423
10.1.1 Consensus in Measurements 424
10.1.2 Consensus in Estimates 429
10.2 Optimal Fusion Filtering under Correlated Noise 433
10.2.1 Error Covariances under Cross Correlation 436
10.3 Hybrid Kalman/UFIR Filter Structures 438
10.3.1 Fusing Estimates with Probabilistic Weights 438
10.3.2 Fusing Kalman and Weighted UFIR Estimates 442
10.4 Estimation under Delayed and Missing Data 444
10.4.1 Deterministic Delays and Missing Data 445
10.4.2 Randomly Delayed and Missing Data 449
10.5 Summary 453
10.6 Problems 454
11 Applications of FIR State Estimators 457
11.1 UFIR Filtering and Prediction of Clock States 458
11.1.1 Clock Model 458
11.1.2 Clock State Estimation over GPS-Based TIE Data 459
11.1.3 Master Clock Error Prediction 460
11.2 Suboptimal Clock Synchronization 463
11.2.1 Clock Digital Synchronization Loop 463
11.3 Localization over WSNs Using Particle/UFIR Filter 468
11.3.1 Sample Impoverishment Issue 470
11.3.2 Hybrid Particle/UFIR Filter 471
11.4 Self-localization over RFID Tag Grids 473
11.4.1 State Space Localization Problem 474
11.4.2 Localization Performance 476
11.5 INS/UWB-Based Quadrotor Localization 478
11.5.1 Quadrotor State Space Model under CMN 479
11.5.2 Localization Performance 481
11.6 Processing of Biosignals 481
11.6.1 ECG Signal Denoising using UFIR Smoothing 482
11.6.2 EMG Envelope Extraction using UFIR Filter 484
11.7 Summary 487
11.8 Problems 488
A Matrix Forms and Relationships 489
A.1 Derivatives 489
A.2 Matrix Identities 489
A.3 Special Matrices 490
A.4 Equations and Inequalities 491
A.5 Linear Matrix Inequalities 493
B Norms 495
B.1 Vector Norms 495
B.2 Matrix Norms 496
B.3 Signal Norms 497
B.4 System Norms 499
References 501
1
Introduction
The limited memory filter appears to be the only device for preventing divergence in the presence of unbounded perturbation.
Andrew H. Jazwinski [79], p. 255
The term state estimation implies that we want to estimate the state of some process, system, or object using its measurements. Since measurements are usually carried out in the presence of noise, we want an accurate and precise estimator, preferably optimal and unbiased. If the environment or data is uncertain (or both) and the system is being attacked by disturbances, we also want the estimator to be robust. Since the estimator usually extracts state from a noisy observation, it is also called a filter, smoother, or predictor. Thus, a state estimator can be represented by a certain block (hardware or software), the operator of which allows transforming (in some sense) input data into an output estimate. Accordingly, the linear state estimator can be designed to have either infinite impulse response (IIR) or finite impulse response (FIR). Since IIR is a feedback effect and FIR is inherent to transversal structures, the properties of such estimators are very different, although both can be represented in batch forms and by iterative algorithms using recursions. Note that effective recursions are available only for delta-correlated (white) noise and errors.
In this chapter, we introduce the reader to FIR and IIR state estimates, discuss cost functions and the most critical properties, and provide a brief historical overview of the most notable works in the area. Since IIR-related recursive Kalman filtering, described in a huge number of outstanding works, serves in a special case of Gaussian noise and diagonal block covariance matrices, our main emphasis will be on the more general FIR approach.
1.1 What Is System State?
When we deal with some stochastic dynamic system or process and want to predict its further behavior, we need to know the system characteristics at the present moment. Thus, we can use the fundamental concept of state variables, a set of which mathematically describes the state of a system. The practical need for this was formulated by Jazwinski in [79] as ".the engineer must know what the system is "doing" at any instant of time" and ".the engineer must know the state of his system."
Obviously, the set of state variables should be sufficient to predict the future system behavior, which means that the number of state variables should not be less than practically required. But the number of state variables should also not exceed a reasonable set, because redundancy, ironically, reduces the estimation accuracy due to random and numerical errors. Consequently, the number of useful state variables is usually small, as will be seen next.
When tracking and localizing mechanical systems, the coordinates of location and velocities in each of the Cartesian coordinates are typical state variables. In precise satellite navigation systems, the coordinates, velocities, and accelerations in each of the Cartesian coordinates are a set of nine state variables. In electrical and electronic systems, the number of state variables is determined by the order of the differential equation or the number of storage elements, which are inductors and capacitors.
In periodic systems, the amplitude, frequency, and phase of the spectral components are necessary state variables. But in clocks that are driven by oscillators (periodic systems), the standard state variables are the time error, fractional frequency offset, and linear frequency drift rate.
In thermodynamics, a set of state variables consists of independent variables of a state function such as internal energy, enthalpy, and entropy. In ecosystem models, typical state variables are the population sizes of plants, animals, and resources. In complex computer systems, various states can be assigned to represent processes.
In industrial control systems, the number of required state variables depends on the plant program and the installation complexity. Here, a state observer provides an estimate of the set of internal plant states based on measurements of its input and output, and a set of state variables is assigned depending on practical applications.
1.1.1 Why and How Do We Estimate State?
The need to know the system state is dictated by many practical problems. An example of signal processing is system identification over noisy input and output. Control systems are stabilized using state feedback. When such problems arise, we need some kind of model and an estimator.
Any stochastic dynamic system can be represented by the first-order linear or nonlinear vector differential equation (in continuous time) or difference equation (in discrete time) with respect to a set of its states. Such equations are called state equations, where state variables are usually affected by internal noise and external disturbances, and the model can be uncertain.
Estimating the state of a system with random components represented by the state equation means evaluating the state approximately using measurements over a finite time interval or all available data. In many cases, the complete set of system states cannot be determined by direct measurements in view of the practical inability of doing so. But even if it is possible, measurements are commonly accompanied by various kinds of noise and errors. Typically, the full set of state variables is observed indirectly by way of the system output, and the observed state is represented with an observation equation, where the measurements are usually affected by internal noise and external disturbances. The important thing is that if the system is observable, then it is possible to completely reconstruct the state of the system from its output measurements using a state observer. Otherwise, when the inner state cannot be observed, many practical problems cannot be solved.
1.1.2 What Model to Estimate State?
Systems and processes can be both nonlinear or linear. Accordingly, we recognize nonlinear and linear state-space models. Linear models are represented by linear equations and Gaussian noise. A model is said to be nonlinear if it is represented by nonlinear equations or linear equations with non-Gaussian random components.
Nonlinear Systems
A physical nonlinear system with random components can be represented in continuous time by the following time-varying state space model,
(1.1) (1.2)where the nonlinear differential equation (1.1) is called the state equation and an algebraic equation 1.2 the observation equation. Here, is the system state vector; , is the input (control) vector; is the state observation vector, is some system error, noise, or disturbance; is an observation error or measurement noise; is a nonlinear system function; and is a nonlinear observation function. Vectors and can be Gaussian or non-Gaussian, correlated or noncorrelated, additive or multiplicative. For time-invariant systems, both nonlinear functions become constant.
In discrete time , a nonlinear system can be represented in state space with a time step using either the forward Euler (FE) method or the backward Euler (BE) method. By the FE method, the discrete-time state equation turns out to be predictive and we have
(1.3) (1.4)where is the state, is the input, is the observation, is the system error or disturbance, and is the observation error. The model in (1.3) and (1.4) is basic for digital control systems, because it matches the predicted estimate required for feedback and model predictive control.
By the BE method, the discrete-time nonlinear state-space model becomes
(1.5) (1.6)to suit the many signal processing problem when prediction is not required. Since the model in (1.5) and (1.6) is not predictive, it usually approximate a nonlinear process more accurately.
Linear Systems
A linear time-varying (LTV) physical system with random components can be represented in continuous time using the following state space model
(1.7) (1.8)where the noise vectors and can be either Gaussian or not, correlated or not. If and are both zero mean, uncorrelated, and white Gaussian with the covariances and , where and are the relevant power spectral densities, then the model in (1.7) and (1.8) is said to be linear. Otherwise, it is nonlinear. Note that all matrices in (1.7) and (1.8) become constant as , , , , when a system is linear time-invariant (LTI). If the order of the disturbance is less than the order of the system, then , and the model in (1.7) and (1.8) becomes standard for problems considering vectors and as the system and measurement noise, respectively.
By the FE method, the linear discrete-time state equation also turns out to be predictive, and the state-space model becomes
(1.9) (1.10)where , , , , and are time-varying matrices. If the discrete noise...
