Discrete-time Stochastic Systems

1. Introduction.- 1.1 What is a Stochastic System?.- Bibhography.- 2. Some Probability Theory.- 2.1 Introduction.- 2.2 Random Variables and Distributions.- 2.3 Conditional Distributions.- 2.4 The Conditional Mean for Gaussian Variables.- 2.5 Complex-Valued Gaussian Variables.- Exercises.- 3. Models.- 3.1 Introduction.- 3.2 Stochastic Processes.- 3.3 Markov Processes and the Concept of State.- 3.4 Covariance Function and Spectrum.- 3.5 Bispectrum.- 3.A Appendix. Linear Complex-Valued Signals and Systems.- 3.B Appendix. Markov Chains.- Exercises.- 4. Analysis.- 4.1 Introduction.- 4.2 Linear Filtering.- 4.3 Spectral Factorization.- 4.4 Continuous-time Models.- 4.5 Sampling Stochastic Models.- 4.6 The Positive Real Part of the Spectrum.- 4.7 Effect of Linear Filtering on the Bispectrum.- 4.8 Algorithms for Covariance Calculations and Sampling.- 4. A Appendix. Auxiliary Lemmas.- Exercises.- 5. Optimal Estimation.- 5.1 Introduction.- 5.2 The Conditional Mean.- 5.3 The Linear Least Mean Square Estimate.- 5.4 Propagation of the Conditional Probability Density Function.- 5.5 Relation to Maximum Likelihood Estimation.- 5.A Appendix. A Lemma for Optimality of the Conditional Mean.- Exercises.- 6. Optimal State Estimation for Linear Systems.- 6.1 Introduction.- 6.2 The Linear Least Mean Square One-Step Prediction and Filter Estimates.- 6.3 The Conditional Mean.- 6.4 Optimal Filtering and Prediction.- 6.5 Smoothing.- 6.6 Maximum a posteriori Estimates.- 6.7 The Stationary Case.- 6.8 Algorithms for Solving the Algebraic Riccati Equation.- 6.A Appendix. Proofs.- Exercises.- 7. Optimal Estimation for Linear Systems by Polynomial Methods.- 7.1 Introduction.- 7.2 Optimal Prediction.- 7.3 Wiener Filters.- 7.4 Minimum Variance Filters.- 7.5 Robustness Against Modelling Errors.- Exercises.-8. Illustration of Optimal Linear Estimation.- 8.1 Introduction.- 8.2 Spectral Factorization.- 8.3 Optimal Prediction.- 8.4 Optimal Filtering.- 8.5 Optimal Smoothing.- 8.6 Estimation Error Variance.- 8.7 Weighting Pattern.- 8.8 Frequency Characteristics.- Exercises.- 9. Nonlinear Filtering.- 9.1 Introduction.- 9.2 Extended Kaiman Filters.- 9.3 Gaussian Sum Estimators.- 9.4 The Multiple Model Approach.- 9.5 Monte Carlo Methods for Propagating the Conditional Probability Density Functions.- 9.6 Quantized Measurements.- 9.7 Median Filters.- 9.A Appendix. Auxiliary results.- Exercises.- 10. Introduction to Optimal Stochastic Control.- 10.1 Introduction.- 10.2 Some Simple Examples.- 10.3 Mathematical Preliminaries.- 10.4 Dynamic Programming.- 10.5 Some Stochastic Controllers.- Exercises.- 11. Linear Quadratic Gaussian Control.- 11.1 Introduction.- 11.2 The Optimal Controllers.- 11.3 Duality Between Estimation and Control.- 11.4 Closed Loop System Properties.- 11.5 Linear Quadratic Gaussian Design by Polynomial Methods.- 11.6 Controller Design by Linear Quadratic Gaussian Theory.- 11. A Appendix. Derivation of the Optimal Linear Quadratic Gaussian Feedback and the Riccati Equation from the Bellman Equation.- Exercises.- Answers to Selected Exercises.