Computational Statistics

PREFACE xv ACKNOWLEDGMENTS xvii 1 REVIEW 1 1.1 Mathematical Notation 1 1.2 Taylor's Theorem and Mathematical Limit Theory 2 1.3 Statistical Notation and Probability Distributions 4 1.4 Likelihood Inference 9 1.5 Bayesian Inference 11 1.6 Statistical Limit Theory 13 1.7 Markov Chains 14 1.8 Computing 17 PART I OPTIMIZATION 2 OPTIMIZATION AND SOLVING NONLINEAR EQUATIONS 21 2.1 Univariate Problems 22 2.2 Multivariate Problems 34 Problems 54 3 COMBINATORIAL OPTIMIZATION 59 3.1 Hard Problems and NP-Completeness 59 3.2 Local Search 65 3.3 Simulated Annealing 68 3.4 Genetic Algorithms 75 3.5 Tabu Algorithms 85 Problems 92 4 EM OPTIMIZATION METHODS 97 4.1 Missing Data, Marginalization, and Notation 97 4.2 The EM Algorithm 98 4.3 EM Variants 111 Problems 121 PART II INTEGRATION AND SIMULATION 5 NUMERICAL INTEGRATION 129 5.1 Newton-Côtes Quadrature 129 5.2 Romberg Integration 139 5.3 Gaussian Quadrature 142 5.4 Frequently Encountered Problems 146 Problems 148 6 SIMULATION AND MONTE CARLO INTEGRATION 151 6.1 Introduction to the Monte Carlo Method 151 6.2 Exact Simulation 152 6.3 Approximate Simulation 163 6.4 Variance Reduction Techniques 180 Problems 195 7 MARKOV CHAIN MONTE CARLO 201 7.1 Metropolis-Hastings Algorithm 202 7.2 Gibbs Sampling 209 7.3 Implementation 218 Problems 230 8 ADVANCED TOPICS IN MCMC 237 8.1 Adaptive MCMC 237 8.2 Reversible Jump MCMC 250 8.3 Auxiliary Variable Methods 256 8.4 Other Metropolis-Hastings Algorithms 260 8.5 Perfect Sampling 264 8.6 Markov Chain Maximum Likelihood 268 8.7 Example: MCMC for Markov Random Fields 269 Problems 279 PART III BOOTSTRAPPING 9 BOOTSTRAPPING 287 9.1 The Bootstrap Principle 287 9.2 Basic Methods 288 9.3 Bootstrap Inference 292 9.4 Reducing Monte Carlo Error 302 9.5 Bootstrapping Dependent Data 303 9.6 Bootstrap Performance 315 9.7 Other Uses of the Bootstrap 316 9.8 Permutation Tests 317 Problems 319 PART IV DENSITY ESTIMATION AND SMOOTHING 10 NONPARAMETRIC DENSITY ESTIMATION 325 10.1 Measures of Performance 326 10.2 Kernel Density Estimation 327 10.3 Nonkernel Methods 341 10.4 Multivariate Methods 345 Problems 359 11 BIVARIATE SMOOTHING 363 11.1 Predictor-Response Data 363 11.2 Linear Smoothers 365 11.3 Comparison of Linear Smoothers 377 11.4 Nonlinear Smoothers 379 11.5 Confidence Bands 384 11.6 General Bivariate Data 388 Problems 389 12 MULTIVARIATE SMOOTHING 393 12.1 Predictor-Response Data 393 12.2 General Multivariate Data 413 Problems 416 DATA ACKNOWLEDGMENTS 421 REFERENCES 423 INDEX 457