Applied Bayesian Modelling

Preface xi 1 Bayesian methods and Bayesian estimation 1 1.1 Introduction 1 1.1.1 Summarising existing knowledge: Prior densities for parameters 2 1.1.2 Updating information: Prior, likelihood and posterior densities 3 1.1.3 Predictions and assessment 5 1.1.4 Sampling parameters 6 1.2 MCMC techniques: The Metropolis-Hastings algorithm 7 1.2.1 Gibbs sampling 8 1.2.2 Other MCMC algorithms 9 1.2.3 INLA approximations 10 1.3 Software for MCMC: BUGS, JAGS and R-INLA 11 1.4 Monitoring MCMC chains and assessing convergence 19 1.4.1 Convergence diagnostics 20 1.4.2 Model identifiability 21 1.5 Model assessment 23 1.5.1 Sensitivity to priors 23 1.5.2 Model checks 24 1.5.3 Model choice 25 References 28 2 Hierarchical models for related units 34 2.1 Introduction: Smoothing to the hyper population 34 2.2 Approaches to model assessment: Penalised fit criteria, marginal likelihood and predictive methods 35 2.2.1 Penalised fit criteria 36 2.2.2 Formal model selection using marginal likelihoods 37 2.2.3 Estimating model probabilities or marginal likelihoods in practice 38 2.2.4 Approximating the posterior density 40 2.2.5 Model averaging from MCMC samples 42 2.2.6 Predictive criteria for model checking and selection: Cross-validation 46 2.2.7 Predictive checks and model choice using complete data replicate sampling 50 2.3 Ensemble estimates: Poisson-gamma and Beta-binomial hierarchical models 53 2.3.1 Hierarchical mixtures for poisson and binomial data 54 2.4 Hierarchical smoothing methods for continuous data 61 2.4.1 Priors on hyperparameters 62 2.4.2 Relaxing normality assumptions 63 2.4.3 Multivariate borrowing of strength 65 2.5 Discrete mixtures and dirichlet processes 69 2.5.1 Finite mixture models 69 2.5.2 Dirichlet process priors 72 2.6 General additive and histogram smoothing priors 78 2.6.1 Smoothness priors 79 2.6.2 Histogram smoothing 80 Exercises 83 Notes 86 References 89 3 Regression techniques 97 3.1 Introduction: Bayesian regression 97 3.2 Normal linear regression 98 3.2.1 Linear regression model checking 99 3.3 Simple generalized linear models: Binomial, binary and Poisson regression 102 3.3.1 Binary and binomial regression 102 3.3.2 Poisson regression 105 3.4 Augmented data regression 107 3.5 Predictor subset choice 110 3.5.1 The g-prior approach 114 3.5.2 Hierarchical lasso prior methods 116 3.6 Multinomial, nested and ordinal regression 126 3.6.1 Nested logit specification 128 3.6.2 Ordinal outcomes 130 Exercises 136 Notes 138 References 144 4 More advanced regression techniques 149 4.1 Introduction 149 4.2 Departures from linear model assumptions and robust alternatives 149 4.3 Regression for overdispersed discrete outcomes 154 4.3.1 Excess zeroes 157 4.4 Link selection 160 4.5 Discrete mixture regressions for regression and outlier status 161 4.5.1 Outlier accommodation 163 4.6 Modelling non-linear regression effects 167 4.6.1 Smoothness priors for non-linear regression 167 4.6.2 Spline regression and other basis functions 169 4.6.3 Priors on basis coefficients 171 4.7 Quantile regression 175 Exercises 177 Notes 177 References 179 5 Meta-analysis and multilevel models 183 5.1 Introduction 183 5.2 Meta-analysis: Bayesian evidence synthesis 184 5.2.1 Common forms of meta-analysis 185 5.2.2 Priors for stage 2 variation in meta-analysis 188 5.2.3 Multivariate meta-analysis 193 5.3 Multilevel models: Univariate continuous outcomes 195 5.4 Multilevel discrete responses 201 5.5 Modelling heteroscedasticity 204 5.6 Multilevel data on multivariate indices 206 Exercises 208 Notes 210 References 211 6 Models for time series 215 6.1 Introduction 215 6.2 Autoregressive and moving average models 216 6.2.1 Dependent errors 218 6.2.2 Bayesian priors in ARMA models 218 6.2.3 Further types of time dependence 222 6.3 Discrete outcomes 229 6.3.1 INAR models for counts 231 6.3.2 Evolution in conjugate process parameters 232 6.4 Dynamic linear and general linear models 235 6.4.1 Further forms of dynamic models 238 6.5 Stochastic variances and stochastic volatility 244 6.5.1 ARCH and GARCH models 244 6.5.2 State space stochastic volatility models 245 6.6 Modelling structural shifts 248 6.6.1 Level, trend and variance shifts 249 6.6.2 Latent state models including historic dependence 250 6.6.3 Switching regressions and autoregressions 251 Exercises 258 Notes 261 References 265 7 Analysis of panel data 273 7.1 Introduction 273 7.2 Hierarchical longitudinal models for metric data 274 7.2.1 Autoregressive errors 275 7.2.2 Dynamic linear models 276 7.2.3 Extended time dependence 276 7.3 Normal linear panel models and normal linear growth curves 278 7.3.1 Growth curves 280 7.3.2 Subject level autoregressive parameters 283 7.4 Longitudinal discrete data: Binary, categorical and Poisson panel data 285 7.4.1 Binary panel data 285 7.4.2 Ordinal panel data 288 7.4.3 Panel data for counts 292 7.5 Random effects selection 295 7.6 Missing data in longitudinal studies 297 Exercises 302 Notes 303 References 306 8 Models for spatial outcomes and geographical association 312 8.1 Introduction 312 8.2 Spatial regressions and simultaneous dependence 313 8.2.1 Regression with localised dependence 316 8.2.2 Binary outcomes 317 8.3 Conditional prior models 321 8.3.1 Ecological analysis involving count data 324 8.4 Spatial covariation and interpolation in continuous space 329 8.4.1 Discrete convolution processes 332 8.5 Spatial heterogeneity and spatially varying coefficient priors 337 8.5.1 Spatial expansion and geographically weighted regression 338 8.5.2 Spatially varying coefficients via multivariate priors 339 8.6 Spatio-temporal models 343 8.6.1 Conditional prior representations 345 8.7 Clustering in relation to known centres 348 8.7.1 Areas or cases as data 350 8.7.2 Multiple sources 350 Exercises 352 Notes 354 References 355 9 Latent variable and structural equation models 364 9.1 Introduction 364 9.2 Normal linear structural equation models 365 9.2.1 Cross-sectional normal SEMs 365 9.2.2 Identifiability constraints 367 9.3 Dynamic factor models, panel data factor models and spatial factor models 372 9.3.1 Dynamic factor models 372 9.3.2 Linear SEMs for panel data 374 9.3.3 Spatial factor models 378 9.4 Latent trait and latent class analysis for discrete outcomes 381 9.4.1 Latent trait models 381 9.4.2 Latent class models 382 9.5 Latent trait models for multilevel data 387 9.6 Structural equation models for missing data 389 Exercises 392 Notes 394 References 397 10 Survival and event history models 402 10.1 Introduction 402 10.2 Continuous time functions for survival 403 10.2.1 Parametric hazard models 405 10.2.2 Semi-parametric hazards 408 10.3 Accelerated hazards 411 10.4 Discrete time approximations 413 10.4.1 Discrete time hazards regression 415 10.5 Accounting for frailty in event history and survival models 417 10.6 Further applications of frailty models 421 10.7 Competing risks 423 Exercises 425 References 426 Index 431