Regression Estimators
A Comparative Study
2nd Edition
Published on 25. August 2010
Book
Hardback
424 pages
978-0-8018-9426-8 (ISBN)
Description
An examination of mathematical formulations of ridge-regression-type estimators points to a curious observation: estimators can be derived by both Bayesian and Frequentist methods. In this updated and expanded edition of his 1990 treatise on the subject, Marvin H. J. Gruber presents, compares, and contrasts the development and properties of ridge-type estimators from these two philosophically different points of view. The book is organized into five sections. Part I gives a historical survey of the literature and summarizes basic ideas in matrix theory and statistical decision theory. Part II explores the mathematical relationships between estimators from both Bayesian and Frequentist points of view. Part III considers the efficiency of estimators with and without averaging over a prior distribution. Part IV applies the methods and results discussed in the previous two sections to the Kalman Filter, analysis of variance models, and penalized splines. Part V surveys recent developments in the field. These include efficiencies of ridge-type estimators for loss functions other than squared error loss functions and applications to information geometry.
Gruber also includes an updated historical survey and bibliography. With more than 150 exercises, Regression Estimators is a valuable resource for graduate students and professional statisticians.
Gruber also includes an updated historical survey and bibliography. With more than 150 exercises, Regression Estimators is a valuable resource for graduate students and professional statisticians.
Reviews / Votes
"A comprehensive treatment... valuable to statisticians who would like to know more about the analytical properties of ridge-type estimators." - Journal of the American Statistical Association "Highly recommended to anyone working on advanced applications or research in estimation in linear models." - Technometrics"More details
Edition
second edition
Language
English
Place of publication
Baltimore, MD
United States
Target group
College/higher education
Edition type
Revised edition
Illustrations
5 s/w Zeichnungen
5 Line drawings, black and white
Dimensions
Height: 229 mm
Width: 152 mm
Thickness: 29 mm
Weight
680 gr
ISBN-13
978-0-8018-9426-8 (9780801894268)
DOI
10.56021/9780801894268
Copyright in bibliographic data and cover images is held by Nielsen Book Services Limited or by the publishers or by their respective licensors: all rights reserved.
Schweitzer Classification
Person
Marvin H. J. Gruber is a professor of mathematics and statistics at the Rochester Institute of Technology.
Content
Preface
Part I: Introduction and Mathematical Preliminaries
1. Introduction
1.1. The Purpose of This Book
1.2. Least Square Estimators and the Need for Alternatives
1.3. Historical Survey
1.4. The Structure of the Book
2. Mathematical and Statistical Preliminaries
2.0. Introduction
2.1. Matrix Theory Results
2.2. The Bayes Estimator (BE)
2.3. Admissible Estimators
2.4. The Minimax Estimator
2.5. Criterion for Comparing Estimators: Theobald's 1974 Result
2.6. Some Useful Inequalities: Some Miscellaneous Useful Matrix Results
2.7. Summary
Part II: The Estimators, Their Derivations, and Their Relationships
3. The Estimators
3.0. The Least Square Estimator and Its Properties
3.1. The Generalized Ridge Regression Estimator
3.2. The Mixed Estimators
3.3. The Linear Minimax Estimator
3.4. The Bayes Estimator
3.6. Summary
4. How the Different Estimators Are Related
4.0. Introduction
4.1. Alternative Forms of the Bayes Estimator Full-Rank Case
4.2. Alternative Forms of the Bayes Estimator Non-Full-Rank Case Estimable Parametric Functions
4.3. Equivalence of the Generalized Ridge Estimator and the BayesEstimator
4.4. Equivalence of the Mixed Estimator and the Bayes Estimator
4.5. Ridge Estimators in the Literature as Special Cases of the BE, Minimax Estimators, or Mixed Estimators
4.6. An Extension of the Gauss-Markov Theorem
4.7. Generalities
4.8. Summary
Part III: Comparing the Efficiency of the Estimators
5. Measures of Efficiency of the Estimators
5.0. Introduction
5.1. The Different Kinds of Mean Square Error
5.2. Zellner's Balanced Loss Function
5.3. The LINEX Loss Function
5.4. Linear Admissibility
5.5. Summary
6. The Average Mean Square Error
6.0. Introduction
6.1. The Forms of the MSE for the Minimax, Bayes, and Mixed Estimators
6.2. The Relationship between the Average Variance and the MSE
6.3. The Average MSE of the Bayes Estimator
6.4. Alternative Forms of the MSE of the Mixed Estimator
6.5. Comparison of the MSE of Different BEs
6.6. Comparison of the MSE of the Ridge and Contraction Estimators
6.7. Comparison of the Average MSE of the Two-Parameter Liu Estimator and the Ordinary Ridge Regression Estimator
6.8. Summary
7. The MSE Neglecting the Prior Assumptions
7.0. Introduction
7.1. The MSE of the BE
7.2. The MSE of the Mixed Estimators Neglecting PriorAssumptions
7.3. Comparison of the Conditional MSE of the Bayes and Least Square Estimators and Comparison of the Conditional and Average MSE
7.4. Comparison of the MSE of a Mixed Estimator with That of the LS Estimators
7.5. Comparison of the MSE of Two Bayes Estimators
7.6. Summary
8. The MSE for Incorrect Prior Assumptions
8.0. Introduction
8.1. The Bayes Estimator and Its MSE
8.2. The Minimax Estimator
8.3. The Mixed Estimator
8.4. Contaminated Priors
8.5. Contaminated (Mixed) Bayes Estimators
8.6. Summary
Part IV: Applications
9. The Kalman Filter
9.0. Introduction
9.1. The Kalman Filter as a Bayes Estimator
9.2. The Kalman Filter as a Recursive Least Square Estimator,and the Connection with the Mixed Estimator
9.3. The Minimax Estimator
9.4. The Generalized Ridge Estimator
9.5. The Average Mean Square Error
9.6. The MSE for Incorrect Initial Prior Assumptions
9.7. Applications
9.8. Recursive Ridge Regression
9.9. Summary
10. Experimental Design Models
10.0. Introduction
10.1. The One-Way ANOVA Model
10.2. The Bayes and Empirical Bayes Estimators
10.3. The Two- Way Classification
10.4. The Bayes and Empirical Bayes Estimators
10.5. Summary
Appendix to Section 10.2. Calculation of the MSE of Section 10.2
11. How Penalized Splines and Ridge- Type EstimatorsAre Related
11.0. Introduction
11.1. Splines as a Special Kind of Regression Model
11.2. Penalized Splines
11.3. The Best Linear Unbiased Predictor (BLUP)
11.4. Two Examples
11.5. Summary
Part V: Alternative Measures of Efficiency
12. Estimation Using Zellner's Balanced Loss Function
12.0. Introduction
12.1. Zellner's Balanced Loss Function
12.2. The Estimators from Different Points of View
12.3. The Average Mean Square Error
12.4. The Risk without Averaging over a Prior Distribution
12.5. Some Optimal Ridge Estimators
12.6. Summary
13. The LINEX and Other Asymmetric Loss Functions
13.0. Introduction
13.1. The LINEX Loss Function
13.2. The Bayes Risk for a Regression Estimator
13.3. The Frequentist Risk
13.4. Summary
14. Distances between Ridge-Type Estimators, andInformation Geometry
14.0. Introduction
14.1. The Relevant Differential Geometry
14.2. The Distance between Two Linear Bayes Estimators, Based on the Prior Distributions
14.3. The Distance between Distributions of Ridge-Type Estimators from a Non-Bayesian Point of View
14.4. Distances between the Mixed Estimators
14.5. An Example Using the Kalman Filter
14.6. Summary
References
Author Index
Subject Index
Part I: Introduction and Mathematical Preliminaries
1. Introduction
1.1. The Purpose of This Book
1.2. Least Square Estimators and the Need for Alternatives
1.3. Historical Survey
1.4. The Structure of the Book
2. Mathematical and Statistical Preliminaries
2.0. Introduction
2.1. Matrix Theory Results
2.2. The Bayes Estimator (BE)
2.3. Admissible Estimators
2.4. The Minimax Estimator
2.5. Criterion for Comparing Estimators: Theobald's 1974 Result
2.6. Some Useful Inequalities: Some Miscellaneous Useful Matrix Results
2.7. Summary
Part II: The Estimators, Their Derivations, and Their Relationships
3. The Estimators
3.0. The Least Square Estimator and Its Properties
3.1. The Generalized Ridge Regression Estimator
3.2. The Mixed Estimators
3.3. The Linear Minimax Estimator
3.4. The Bayes Estimator
3.6. Summary
4. How the Different Estimators Are Related
4.0. Introduction
4.1. Alternative Forms of the Bayes Estimator Full-Rank Case
4.2. Alternative Forms of the Bayes Estimator Non-Full-Rank Case Estimable Parametric Functions
4.3. Equivalence of the Generalized Ridge Estimator and the BayesEstimator
4.4. Equivalence of the Mixed Estimator and the Bayes Estimator
4.5. Ridge Estimators in the Literature as Special Cases of the BE, Minimax Estimators, or Mixed Estimators
4.6. An Extension of the Gauss-Markov Theorem
4.7. Generalities
4.8. Summary
Part III: Comparing the Efficiency of the Estimators
5. Measures of Efficiency of the Estimators
5.0. Introduction
5.1. The Different Kinds of Mean Square Error
5.2. Zellner's Balanced Loss Function
5.3. The LINEX Loss Function
5.4. Linear Admissibility
5.5. Summary
6. The Average Mean Square Error
6.0. Introduction
6.1. The Forms of the MSE for the Minimax, Bayes, and Mixed Estimators
6.2. The Relationship between the Average Variance and the MSE
6.3. The Average MSE of the Bayes Estimator
6.4. Alternative Forms of the MSE of the Mixed Estimator
6.5. Comparison of the MSE of Different BEs
6.6. Comparison of the MSE of the Ridge and Contraction Estimators
6.7. Comparison of the Average MSE of the Two-Parameter Liu Estimator and the Ordinary Ridge Regression Estimator
6.8. Summary
7. The MSE Neglecting the Prior Assumptions
7.0. Introduction
7.1. The MSE of the BE
7.2. The MSE of the Mixed Estimators Neglecting PriorAssumptions
7.3. Comparison of the Conditional MSE of the Bayes and Least Square Estimators and Comparison of the Conditional and Average MSE
7.4. Comparison of the MSE of a Mixed Estimator with That of the LS Estimators
7.5. Comparison of the MSE of Two Bayes Estimators
7.6. Summary
8. The MSE for Incorrect Prior Assumptions
8.0. Introduction
8.1. The Bayes Estimator and Its MSE
8.2. The Minimax Estimator
8.3. The Mixed Estimator
8.4. Contaminated Priors
8.5. Contaminated (Mixed) Bayes Estimators
8.6. Summary
Part IV: Applications
9. The Kalman Filter
9.0. Introduction
9.1. The Kalman Filter as a Bayes Estimator
9.2. The Kalman Filter as a Recursive Least Square Estimator,and the Connection with the Mixed Estimator
9.3. The Minimax Estimator
9.4. The Generalized Ridge Estimator
9.5. The Average Mean Square Error
9.6. The MSE for Incorrect Initial Prior Assumptions
9.7. Applications
9.8. Recursive Ridge Regression
9.9. Summary
10. Experimental Design Models
10.0. Introduction
10.1. The One-Way ANOVA Model
10.2. The Bayes and Empirical Bayes Estimators
10.3. The Two- Way Classification
10.4. The Bayes and Empirical Bayes Estimators
10.5. Summary
Appendix to Section 10.2. Calculation of the MSE of Section 10.2
11. How Penalized Splines and Ridge- Type EstimatorsAre Related
11.0. Introduction
11.1. Splines as a Special Kind of Regression Model
11.2. Penalized Splines
11.3. The Best Linear Unbiased Predictor (BLUP)
11.4. Two Examples
11.5. Summary
Part V: Alternative Measures of Efficiency
12. Estimation Using Zellner's Balanced Loss Function
12.0. Introduction
12.1. Zellner's Balanced Loss Function
12.2. The Estimators from Different Points of View
12.3. The Average Mean Square Error
12.4. The Risk without Averaging over a Prior Distribution
12.5. Some Optimal Ridge Estimators
12.6. Summary
13. The LINEX and Other Asymmetric Loss Functions
13.0. Introduction
13.1. The LINEX Loss Function
13.2. The Bayes Risk for a Regression Estimator
13.3. The Frequentist Risk
13.4. Summary
14. Distances between Ridge-Type Estimators, andInformation Geometry
14.0. Introduction
14.1. The Relevant Differential Geometry
14.2. The Distance between Two Linear Bayes Estimators, Based on the Prior Distributions
14.3. The Distance between Distributions of Ridge-Type Estimators from a Non-Bayesian Point of View
14.4. Distances between the Mixed Estimators
14.5. An Example Using the Kalman Filter
14.6. Summary
References
Author Index
Subject Index