Case Studies in Bayesian Statistical Modelling and Analysis

Preface xvii List of contributors xix 1 Introduction 1 Clair L. Alston, Margaret Donald, Kerrie L. Mengersen and Anthony N. Pettitt 1.1 Introduction 1 1.2 Overview 1 1.3 Further reading 8 1.3.1 Bayesian theory and methodology 8 1.3.2 Bayesian methodology 10 1.3.3 Bayesian computation 10 1.3.4 Bayesian software 11 1.3.5 Applications 13 References 13 2 Introduction to MCMC 17 Anthony N. Pettitt and Candice M. Hincksman 2.1 Introduction 17 2.2 Gibbs sampling 18 2.2.1 Example: Bivariate normal 18 2.2.2 Example: Change-point model 19 2.3 Metropolis-Hastings algorithms 19 2.3.1 Example: Component-wise MH or MH within Gibbs 20 2.3.2 Extensions to basic MCMC 21 2.3.3 Adaptive MCMC 22 2.3.4 Doubly intractable problems 22 2.4 Approximate Bayesian computation 24 2.5 Reversible jump MCMC 25 2.6 MCMC for some further applications 26 References 27 3 Priors: Silent or active partners of Bayesian inference? 30 Samantha Low Choy 3.1 Priors in the very beginning 30 3.1.1 Priors as a basis for learning 32 3.1.2 Priors and philosophy 32 3.1.3 Prior chronology 33 3.1.4 Pooling prior information 34 3.2 Methodology I: Priors defined by mathematical criteria 35 3.2.1 Conjugate priors 35 3.2.2 Impropriety and hierarchical priors 37 3.2.3 Zellner's g-prior for regression models 37 3.2.4 Objective priors 38 3.3 Methodology II: Modelling informative priors 40 3.3.1 Informative modelling approaches 40 3.3.2 Elicitation of distributions 42 3.4 Case studies 44 3.4.1 Normal likelihood: Time to submit research dissertations 44 3.4.2 Binomial likelihood: Surveillance for exotic plant pests 47 3.4.3 Mixture model likelihood: Bioregionalization 50 3.4.4 Logistic regression likelihood: Mapping species distribution via habitat models 53 3.5 Discussion 57 3.5.1 Limitations 57 3.5.2 Finding out about the problem 58 3.5.3 Prior formulation 59 3.5.4 Communication 60 3.5.5 Conclusion 61 Acknowledgements 61 References 61 4 Bayesian analysis of the normal linear regression model 66 Christopher M. Strickland and Clair L. Alston 4.1 Introduction 66 4.2 Case studies 67 4.2.1 Case study 1: Boston housing data set 67 4.2.2 Case study 2: Production of cars and station wagons 67 4.3 Matrix notation and the likelihood 67 4.4 Posterior inference 68 4.4.1 Natural conjugate prior 69 4.4.2 Alternative prior specifications 73 4.4.3 Generalizations of the normal linear model 74 4.4.4 Variable selection 78 4.5 Analysis 81 4.5.1 Case study 1: Boston housing data set 81 4.5.2 Case study 2: Car production data set 85 References 88 5 Adapting ICU mortality models for local data: A Bayesian approach 90 Petra L. Graham, Kerrie L. Mengersen and David A. Cook 5.1 Introduction 90 5.2 Case study: Updating a known risk-adjustment model for local use 91 5.3 Models and methods 92 5.4 Data analysis and results 96 5.4.1 Updating using the training data 96 5.4.2 Updating the model yearly 98 5.5 Discussion 100 References 101 6 A Bayesian regression model with variable selection for genome-wide association studies 103 Carla Chen, Kerrie L. Mengersen, Katja Ickstadt and Jonathan M. Keith 6.1 Introduction 103 6.2 Case study: Case-control of Type 1 diabetes 104 6.3 Case study: GENICA 105 6.4 Models and methods 105 6.4.1 Main effect models 105 6.4.2 Main effects and interactions 108 6.5 Data analysis and results 109 6.5.1 WTCCC TID 109 6.5.2 GENICA 110 6.6 Discussion 112 Acknowledgements 115 References 115 6.A Appendix: SNP IDs 117 7 Bayesian meta-analysis 118 Jegar O. Pitchforth and Kerrie L. Mengersen 7.1 Introduction 118 7.2 Case study 1: Association between red meat consumption and breast cancer 119 7.2.1 Background 119 7.2.2 Meta-analysis models 121 7.2.3 Computation 125 7.2.4 Results 125 7.2.5 Discussion 129 7.3 Case study 2: Trends in fish growth rate and size 130 7.3.1 Background 130 7.3.2 Meta-analysis models 131 7.3.3 Computation 134 7.3.4 Results 134 7.3.5 Discussion 135 Acknowledgements 137 References 138 8 Bayesian mixed effects models 141 Clair L. Alston, Christopher M. Strickland, Kerrie L. Mengersen and Graham E. Gardner 8.1 Introduction 141 8.2 Case studies 142 8.2.1 Case study 1: Hot carcase weight of sheep carcases 142 8.2.2 Case study 2: Growth of primary school girls 142 8.3 Models and methods 146 8.3.1 Model for Case study 1 147 8.3.2 Model for Case study 2 148 8.3.3 MCMC estimation 149 8.4 Data analysis and results 150 8.5 Discussion 158 References 158 9 Ordering of hierarchies in hierarchical models: Bone mineral density estimation 159 Cathal D. Walsh and Kerrie L. Mengersen 9.1 Introduction 159 9.2 Case study 160 9.2.1 Measurement of bone mineral density 160 9.3 Models 161 9.3.1 Hierarchical model 162 9.3.2 Model H1 163 9.3.3 Model H2 163 9.4 Data analysis and results 164 9.4.1 Model H1 164 9.4.2 Model H2 165 9.4.3 Implication of ordering 166 9.4.4 Simulation study 166 9.4.5 Study design 166 9.4.6 Simulation study results 167 9.5 Discussion 168 References 168 9.A Appendix: Likelihoods 170 10 BayesianWeibull survival model for gene expression data 171 Sri Astuti Thamrin, James M. McGree and Kerrie L. Mengersen 10.1 Introduction 171 10.2 Survival analysis 172 10.3 Bayesian inference for the Weibull survival model 174 10.3.1 Weibull model without covariates 174 10.3.2 Weibull model with covariates 175 10.3.3 Model evaluation and comparison 176 10.4 Case study 178 10.4.1 Weibull model without covariates 178 10.4.2 Weibull survival model with covariates 180 10.4.3 Model evaluation and comparison 182 10.5 Discussion 182 References 183 11 Bayesian change point detection in monitoring clinical outcomes 186 Hassan Assareh, Ian Smith and Kerrie L. Mengersen 11.1 Introduction 186 11.2 Case study: Monitoring intensive care unit outcomes 187 11.3 Risk-adjusted control charts 187 11.4 Change point model 188 11.5 Evaluation 189 11.6 Performance analysis 190 11.7 Comparison of Bayesian estimator with other methods 194 11.8 Conclusion 194 References 195 12 Bayesian splines 197 Samuel Clifford and Samantha Low Choy 12.1 Introduction 197 12.2 Models and methods 197 12.2.1 Splines and linear models 197 12.2.2 Link functions 198 12.2.3 Bayesian splines 198 12.2.4 Markov chain Monte Carlo 204 12.2.5 Model choice 206 12.2.6 Posterior diagnostics 207 12.3 Case studies 207 12.3.1 Data 207 12.3.2 Analysis 208 12.4 Conclusion 216 12.4.1 Discussion 216 12.4.2 Extensions 217 12.4.3 Summary 218 References 218 13 Disease mapping using Bayesian hierarchical models 221 Arul Earnest, Susanna M. Cramb and Nicole M. White 13.1 Introduction 221 13.2 Case studies 224 13.2.1 Case study 1: Spatio-temporal model examining the incidence of birth defects 224 13.2.2 Case study 2: Relative survival model examining survival from breast cancer 225 13.3 Models and methods 225 13.3.1 Case study 1 225 13.3.2 Case study 2 229 13.4 Data analysis and results 230 13.4.1 Case study 1 230 13.4.2 Case study 2 231 13.5 Discussion 234 References 237 14 Moisture, crops and salination: An analysis of a three-dimensional agricultural data set 240 Margaret Donald, Clair L. Alston, Rick Young and Kerrie L. Mengersen 14.1 Introduction 240 14.2 Case study 241 14.2.1 Data 242 14.2.2 Aim of the analysis 242 14.3 Review 243 14.3.1 General methodology 243 14.3.2 Computations 243 14.4 Case study modelling 243 14.4.1 Modelling framework 243 14.5 Model implementation: Coding considerations 246 14.5.1 Neighbourhood matrices and CAR models 246 14.5.2 Design matrices vs indexing 246 14.6 Case study results 247 14.7 Conclusions 249 References 250 15 A Bayesian approach to multivariate state space modelling: A study of a Fama-French asset-pricing model with time-varying regressors 252 Christopher M. Strickland and Philip Gharghori 15.1 Introduction 252 15.2 Case study: Asset pricing in financial markets 253 15.2.1 Data 254 15.3 Time-varying Fama-French model 254 15.3.1 Specific models under consideration 255 15.4 Bayesian estimation 256 15.4.1 Gibbs sampler 256 15.4.2 Sampling epsilon 257 15.4.3 Sampling ß1:n 257 15.4.4 Sampling alpha 259 15.4.5 Likelihood calculation 260 15.5 Analysis 261 15.5.1 Prior elicitation 261 15.5.2 Estimation output 261 15.6 Conclusion 264 References 265 16 Bayesian mixture models: When the thing you need to know is the thing you cannot measure 267 Clair L. Alston, Kerrie L. Mengersen and Graham E. Gardner 16.1 Introduction 267 16.2 Case study: CT scan images of sheep 268 16.3 Models and methods 270 16.3.1 Bayesian mixture models 270 16.3.2 Parameter estimation using the Gibbs sampler 273 16.3.3 Extending the model to incorporate spatial information 274 16.4 Data analysis and results 276 16.4.1 Normal Bayesian mixture model 276 16.4.2 Spatial mixture model 278 16.4.3 Carcase volume calculation 281 16.5 Discussion 284 References 284 17 Latent class models in medicine 287 Margaret Rolfe, Nicole M. White and Carla Chen 17.1 Introduction 287 17.2 Case studies 288 17.2.1 Case study 1: Parkinson's disease 288 17.2.2 Case study 2: Cognition in breast cancer 288 17.3 Models and methods 289 17.3.1 Finite mixture models 290 17.3.2 Trajectory mixture models 292 17.3.3 Goodness of fit 296 17.3.4 Label switching 297 17.3.5 Model computation 298 17.4 Data analysis and results 300 17.4.1 Case study 1: Phenotype identification in PD 300 17.4.2 Case study 2: Trajectory groups for verbal memory 302 17.5 Discussion 306 References 307 18 Hidden Markov models for complex stochastic processes: A case study in electrophysiology 310 Nicole M. White, Helen Johnson, Peter Silburn, Judith Rousseau and Kerrie L. Mengersen 18.1 Introduction 310 18.2 Case study: Spike identification and sorting of extracellular recordings 311 18.3 Models and methods 312 18.3.1 What is an HMM? 312 18.3.2 Modelling a single AP: Application of a simple HMM 313 18.3.3 Multiple neurons: An application of a factorial HMM 315 18.3.4 Model estimation and inference 317 18.4 Data analysis and results 320 18.4.1 Simulation study 320 18.4.2 Case study: Extracellular recordings collected during deep brain stimulation 323 18.5 Discussion 326 References 327 19 Bayesian classification and regression trees 330 Rebecca A. O'Leary, Samantha Low Choy, Wenbiao Hu and Kerrie L. Mengersen 19.1 Introduction 330 19.2 Case studies 332 19.2.1 Case study 1: Kyphosis 332 19.2.2 Case study 2: Cryptosporidium 332 19.3 Models and methods 334 19.3.1 CARTs 334 19.3.2 Bayesian CARTs 335 19.4 Computation 337 19.4.1 Building the BCART model - stochastic search 337 19.4.2 Model diagnostics and identifying good trees 339 19.5 Case studies - results 341 19.5.1 Case study 1: Kyphosis 341 19.5.2 Case study 2: Cryptosporidium 343 19.6 Discussion 345 References 346 20 Tangled webs: Using Bayesian networks in the fight against infection 348 Mary Waterhouse and Sandra Johnson 20.1 Introduction to Bayesian network modelling 348 20.1.1 Building a BN 349 20.2 Introduction to case study 351 20.3 Model 352 20.4 Methods 354 20.5 Results 355 20.6 Discussion 357 References 359 21 Implementing adaptive dose finding studies using sequential Monte Carlo 361 James M. McGree, Christopher C. Drovandi and Anthony N. Pettitt 21.1 Introduction 361 21.2 Model and priors 363 21.3 SMC for dose finding studies 364 21.3.1 Importance sampling 364 21.3.2 SMC 365 21.3.3 Dose selection procedure 367 21.4 Example 369 21.5 Discussion 371 References 372 21.A Appendix: Extra example 373 22 Likelihood-free inference for transmission rates of nosocomial pathogens 374 Christopher C. Drovandi and Anthony N. Pettitt 22.1 Introduction 374 22.2 Case study: Estimating transmission rates of nosocomial pathogens 375 22.2.1 Background 375 22.2.2 Data 376 22.2.3 Objective 376 22.3 Models and methods 376 22.3.1 Models 376 22.3.2 Computing the likelihood 379 22.3.3 Model simulation 380 22.3.4 ABC 381 22.3.5 ABC algorithms 382 22.4 Data analysis and results 384 22.5 Discussion 385 References 386 23 Variational Bayesian inference for mixture models 388 Clare A. McGrory 23.1 Introduction 388 23.2 Case study: Computed tomography (CT) scanning of a loin portion of a pork carcase 390 23.3 Models and methods 392 23.4 Data analysis and results 397 23.5 Discussion 399 References 399 23.A Appendix: Form of the variational posterior for a mixture of multivariate normal densities 401 24 Issues in designing hybrid algorithms 403 Jeong E. Lee, Kerrie L. Mengersen and Christian P. Robert 24.1 Introduction 403 24.2 Algorithms and hybrid approaches 406 24.2.1 Particle system in the MCMC context 407 24.2.2 MALA 407 24.2.3 DRA 408 24.2.4 PS 409 24.2.5 Population Monte Carlo (PMC) algorithm 410 24.3 Illustration of hybrid algorithms 412 24.3.1 Simulated data set 412 24.3.2 Application: Aerosol particle size 415 24.4 Discussion 417 References 418 25 A Python package for Bayesian estimation using Markov chain Monte Carlo 421 Christopher M. Strickland, Robert J. Denham, Clair L. Alston and Kerrie L. Mengersen 25.1 Introduction 421 25.2 Bayesian analysis 423 25.2.1 MCMC methods and implementation 424 25.2.2 Normal linear Bayesian regression model 433 25.3 Empirical illustrations 437 25.3.1 Example 1: Linear regression model - variable selection and estimation 438 25.3.2 Example 2: Loglinear model 441 25.3.3 Example 3: First-order autoregressive regression 446 25.4 Using PyMCMC efficiently 451 25.4.1 Compiling code in Windows 455 25.5 PyMCMC interacting with R 457 25.6 Conclusions 458 25.7 Obtaining PyMCMC 459 References 459 Index 461