Simplicity, Complexity and Modelling

Preface ix Acknowledgements xi Contributing authors xiii 1 Introduction 1 Mike Christie, Andrew Cliffe, Philip Dawid and Stephen Senn 1.1 The origins of the SCAM project 1 1.2 The scope of modelling in the modern world 2 1.3 The different professions and traditions engaged in modelling 3 1.4 Different types of models 3 1.5 Different purposes for modelling 5 1.6 The purpose of the book 6 1.7 Overview of the chapters 6 References 8 2 Statistical model selection 11 Philip Dawid and Stephen Senn 2.1 Introduction 11 2.2 Explanation or prediction? 12 2.3 Levels of uncertainty 12 2.4 Bias-variance trade-off 13 2.5 Statistical models 15 2.5.1 Within-model inference 16 2.6 Model comparison 18 2.7 Bayesian model comparison 18 2.7.1 Model uncertainty 19 2.7.2 Laplace approximation 20 2.8 Penalized likelihood 20 2.8.1 Bayesian information criterion 21 2.9 The Akaike information criterion 21 2.9.1 Inconsistency of AIC 23 2.10 Significance testing 23 2.11 Many variables 27 2.12 Data-driven approaches 28 2.12.1 Cross-validation 29 2.12.2 Prequential analysis 29 2.13 Model selection or model averaging? 30 References 31 3 Modelling in drug development 35 Stephen Senn 3.1 Introduction 35 3.2 The nature of drug development and scope for statistical modelling 36 3.3 Simplicity versus complexity in phase III trials 36 3.3.1 The nature of phase III trials 36 3.3.2 The case for simplicity in analysing phase III trials 37 3.3.3 The case for complexity in modelling clinical trials 38 3.4 Some technical issues 39 3.4.1 The effect of covariate adjustment in linear models 40 3.4.2 The effect of covariate adjustment in non-linear models 42 3.4.3 Random effects in multi-centre trials 44 3.4.4 Subgroups and interactions 45 3.4.5 Bayesian approaches 46 3.5 Conclusion 46 3.6 Appendix: The effect of covariate adjustment on the variance multiplier in least squares 47 References 48 4 Modelling with deterministic computer models 51 Jeremy E. Oakley 4.1 Introduction 51 4.2 Metamodels and emulators for computationally expensive simulators 52 4.2.1 Gaussian processes emulators 53 4.2.2 Multivariate outputs 56 4.3 Uncertainty analysis 57 4.4 Sensitivity analysis 58 4.4.1 Variance-based sensitivity analysis 58 4.4.2 Value of information 61 4.5 Calibration and discrepancy 63 4.6 Discussion 64 References 65 5 Modelling future climates 69 Peter Challenor and Robin Tokmakian 5.1 Introduction 69 5.2 What is the risk from climate change? 70 5.3 Climate models 70 5.4 An anatomy of uncertainty 72 5.4.1 Aleatoric uncertainty 72 5.4.2 Epistemic uncertainty 73 5.5 Simplicity and complexity 75 5.6 An example: The collapse of the thermohaline circulation 77 5.7 Conclusions 79 References 79 6 Modelling climate change impacts for adaptation assessments 83 Suraje Dessai and Jeroen van der Sluijs 6.1 Introduction 83 6.1.1 Climate impact assessment 84 6.2 Modelling climate change impacts: From world development paths to localized impacts 87 6.2.1 Greenhouse gas emissions 87 6.2.2 Climate models 90 6.2.3 Downscaling 93 6.2.4 Regional/local climate change impacts 94 6.3 Discussion 95 6.3.1 Multiple routes of uncertainty assessment 96 6.3.2 What is the appropriate balance between simplicity and complexity? 96 References 98 7 Modelling in water distribution systems 103 Zoran Kapelan 7.1 Introduction 103 7.2 Water distribution system models 104 7.2.1 Water distribution systems 104 7.2.2 WDS hydraulic models 104 7.2.3 Uncertainty in WDS hydraulic modelling 107 7.3 Calibration of WDS hydraulic models 108 7.3.1 Calibration problem 108 7.3.2 Existing approaches 109 7.3.3 Case study 113 7.4 Sampling design for calibration 116 7.4.1 Sampling design problem 116 7.4.2 Existing approaches 116 7.4.3 Case study 120 7.5 Summary and conclusions 120 References 122 8 Modelling for flood risk management 125 Jim Hall 8.1 Introduction 125 8.2 Flood risk management 126 8.2.1 Long-term change 130 8.2.2 Uncertainty 131 8.3 Multi-purpose management 131 8.4 Modelling for flood risk management 132 8.4.1 Source 132 8.4.2 Pathway 132 8.4.3 Receptors 135 8.4.4 An example of a system model: Towyn 135 8.5 Model choice 137 8.6 Conclusions 143 References 144 9 Uncertainty quantification and oil reservoir modelling 147 Mike Christie 9.1 Introduction 147 9.2 Bayesian framework 148 9.2.1 Solution errors 149 9.3 Quantifying uncertainty in prediction of oil recovery 150 9.3.1 Stochastic sampling algorithms 151 9.3.2 Computing uncertainties from multiple history matched models 153 9.4 Inverse problems and reservoir model history matching 155 9.4.1 Synthetic problems 155 9.4.2 Imperial college fault model 157 9.4.3 Comparison of algorithms on a real field example 158 9.5 Selecting appropriate detail in models 162 9.5.1 Adaptive multiscale estimation 162 9.5.2 Bayes factors 165 9.5.3 Application of solution error modelling 167 9.6 Summary 170 References 171 10 Modelling in radioactive waste disposal 173 Andrew Cliffe 10.1 Introduction 173 10.2 The radioactive waste problem 174 10.2.1 What is radioactive waste? 174 10.2.2 How much radioactive waste is there? 175 10.2.3 What are the options for long-term management of radioactive waste? 175 10.3 The treatment of uncertainty in radioactive waste disposal 177 10.3.1 Deep geological disposal 177 10.3.2 Repository performance assessment 177 10.3.3 Modelling 179 10.3.4 Model verification and validation 180 10.3.5 Strategies for dealing with uncertainty 182 10.4 Summary and conclusions 184 References 184 11 Issues for modellers 187 Mike Christie, Andrew Cliffe, Philip Dawid and Stephen Senn 11.1 What are models and what are they useful for? 187 11.2 Appropriate levels of complexity 189 11.3 Uncertainty 190 11.3.1 Model inputs and parameter uncertainty 190 11.3.2 Model uncertainty 191 References 192 Glossary 193 Index 201