Kernel Adaptive Filtering - A Comprehensive Introduction

PREFACE. ACKNOWLEDGMENTS. NOTATION. ABBREVIATIONS AND SYMBOLS. 1 BACKGROUND AND PREVIEW. 1.1 Supervised, Sequential, and Active Learning. 1.2 Linear Adaptive Filters. 1.3 Nonlinear Adaptive Filters. 1.4 Reproducing Kernel Hilbert Spaces. 1.5 Kernel Adaptive Filters. 1.6 Summarizing Remarks. Endnotes. 2 KERNEL LEAST-MEAN-SQUARE ALGORITHM. 2.1 Least-Mean-Square Algorithm. 2.2 Kernel Least-Mean-Square Algorithm. 2.3 Kernel and Parameter Selection. 2.4 Step-Size Parameter. 2.5 Novelty Criterion. 2.6 Self-Regularization Property of KLMS. 2.7 Leaky Kernel Least-Mean-Square Algorithm. 2.8 Normalized Kernel Least-Mean-Square Algorithm. 2.9 Kernel ADALINE. 2.10 Resource Allocating Networks. 2.11 Computer Experiments. 2.12 Conclusion. Endnotes. 3 KERNEL AFFINE PROJECTION ALGORITHMS. 3.1 Affine Projection Algorithms. 3.2 Kernel Affine Projection Algorithms. 3.3 Error Reusing. 3.4 Sliding Window Gram Matrix Inversion. 3.5 Taxonomy for Related Algorithms. 3.6 Computer Experiments. 3.7 Conclusion. Endnotes. 4 KERNEL RECURSIVE LEAST-SQUARES ALGORITHM. 4.1 Recursive Least-Squares Algorithm. 4.2 Exponentially Weighted Recursive Least-Squares Algorithm. 4.3 Kernel Recursive Least-Squares Algorithm. 4.4 Approximate Linear Dependency. 4.5 Exponentially Weighted Kernel Recursive Least-Squares Algorithm. 4.6 Gaussian Processes for Linear Regression. 4.7 Gaussian Processes for Nonlinear Regression. 4.8 Bayesian Model Selection. 4.9 Computer Experiments. 4.10 Conclusion. Endnotes. 5 EXTENDED KERNEL RECURSIVE LEAST-SQUARES ALGORITHM. 5.1 Extended Recursive Least Squares Algorithm. 5.2 Exponentially Weighted Extended Recursive Least Squares Algorithm. 5.3 Extended Kernel Recursive Least Squares Algorithm. 5.4 EX-KRLS for Tracking Models. 5.5 EX-KRLS with Finite Rank Assumption. 5.6 Computer Experiments. 5.7 Conclusion. Endnotes. 6 DESIGNING SPARSE KERNEL ADAPTIVE FILTERS. 6.1 Definition of Surprise. 6.2 A Review of Gaussian Process Regression. 6.3 Computing Surprise. 6.4 Kernel Recursive Least Squares with Surprise Criterion. 6.5 Kernel Least Mean Square with Surprise Criterion. 6.6 Kernel Affine Projection Algorithms with Surprise Criterion. 6.7 Computer Experiments. 6.8 Conclusion. Endnotes. EPILOGUE. APPENDIX. A MATHEMATICAL BACKGROUND. A.1 Singular Value Decomposition. A.2 Positive-Definite Matrix. A.3 Eigenvalue Decomposition. A.4 Schur Complement. A.5 Block Matrix Inverse. A.6 Matrix Inversion Lemma. A.7 Joint, Marginal, and Conditional Probability. A.8 Normal Distribution. A.9 Gradient Descent. A.10 Newton's Method. B. APPROXIMATE LINEAR DEPENDENCY AND SYSTEM STABILITY. REFERENCES. INDEX.