Statistics and the Evaluation of Evidence for Forensic Scientists

Foreword iii Preface to first edition v Preface to second edition ix Preface to third edition xiii 1 Uncertainty in forensic science 1 1.1 Introduction 1 1.2 Statistics and the law 2 1.3 Uncertainty in scientific evidence 5 1.3.1 The frequentist method 7 1.3.2 Stains of body fluids 9 1.3.3 Glass fragments 10 1.4 Terminology 14 1.5 Types of data 16 1.6 Populations 17 1.7 Probability 19 1.7.1 Introduction 19 1.7.2 A standard for uncertainty 22 1.7.3 Events 26 1.7.4 Classical and frequentist definitions of probability and their limitations 27 1.7.5 Subjective definition of probability 28 1.7.6 The quantification of probability through a betting scheme 30 1.7.7 Probabilities and frequencies: the role of exchangeability 32 1.7.8 Laws of probability 37 1.7.9 Dependent events and background information 38 1.7.10 Law of total probability 43 1.7.11 Updating of probabilities 45 2 The evaluation of evidence 49 2.1 Odds 49 2.1.1 Complementary events 49 2.1.2 Examples 50 2.1.3 Definition of odds 51 2.2 Bayes' Theorem 53 2.2.1 Statement of the theorem 53 2.2.2 Examples 53 2.3 The odds form of Bayes' Theorem 59 2.3.1 Likelihood ratio 59 2.3.2 Bayes' factor and likelihood ratio 61 2.3.3 Three-way tables 64 2.3.4 Logarithm of the likelihood ratio 66 2.4 The value of evidence 68 2.4.1 Evaluation of forensic evidence 68 2.4.2 Justification of the use of the likelihood ratio 76 2.4.3 Single value for the likelihood ratio 78 2.4.4 Role of background information 79 2.4.5 Summary of competing propositions 80 2.4.6 Qualitative scale for the value of the evidence 83 2.5 Errors in interpretation 88 2.5.1 Fallacy of the transposed conditional 91 2.5.2 Source probability error 93 2.5.3 Ultimate issue error 95 2.5.4 Defence attorneyÕs fallacy 95 2.5.5 Probability (another match) error 96 2.5.6 Numerical conversion error 97 2.5.7 False positive fallacy 99 2.5.8 Expected value fallacy 100 2.5.9 Uniqueness 101 2.5.10 Other difficulties 102 2.5.11 Empirical evidence of errors in interpretation 108 2.6 Misinterpretations 113 2.7 Explanation of transposed conditional and defence attorney's fallacies 115 2.7.1 Explanation of the fallacy of the transposed conditional 115 2.7.2 Explanation of the false positive fallacy 116 2.7.3 Explanation of the defence attorney's fallacy 118 2.8 Making coherent decisions 119 2.8.1 Elements of statistical decision theory 119 2.8.2 Decision analysis: an example 121 2.9 Graphical probabilistic models: Bayesian networks 123 2.9.1 Elements of Bayesian networks 124 2.9.2 The construction of Bayesian networks 126 2.9.3 Bayesian decision networks (influence diagrams) 131 3 Historical review 135 3.1 Early history 135 3.2 The Dreyfus case 139 3.3 Statistical arguments by early twentieth-century forensic scientists 141 3.4 People v. Collins 144 3.5 Discriminating power 148 3.5.1 Derivation 148 3.5.2 Evaluation of evidence by discriminating power 149 3.5.3 Finite samples 152 3.5.4 Combination of independent systems 154 3.5.5 Correlated attributes 155 3.6 Significance probabilities 157 3.6.1 Calculation of significance probabilities 157 3.6.2 Relationship to likelihood ratio 161 3.6.3 Combination of significance probabilities 164 3.7 Coincidence probabilities 165 3.7.1 Introduction 165 3.7.2 Comparison stage 167 3.7.3 Significance stage 168 3.8 Likelihood ratio 169 4 Bayesian inference 175 4.1 Introduction 175 4.2 Inference for a proportion 179 4.2.1 Interval estimation 181 4.2.2 Estimation with zero occurrences in a sample 186 4.2.3 Uncertainty on sensitivity and specificity 189 4.3 Sampling 191 4.3.1 Choice of sample size in large consignments 195 4.3.2 Choice of sample size in small consignments 201 4.4 Bayesian networks for sampling inspection 206 4.4.1 Large consignments 206 4.4.2 Small consignments 209 4.5 Inference for a Normal mean 211 4.5.1 Known variance 212 4.5.2 Unknown variance 215 4.5.3 Interval estimation 219 4.6 Quantity estimation 221 4.6.1 Predictive approach in small consignments 222 4.6.2 Predictive approach in large consignments 227 4.7 Decision analysis 228 4.7.1 Standard loss functions 229 4.7.2 Decision analysis for forensic sampling 233 5 Evidence and propositions: theory 239 5.1 The choice of propositions and pre-assessment 239 5.2 Levels of propositions and roles of the forensic scientist 240 5.3 The formal development of a likelihood ratio for different propositions and discrete characteristics 245 5.3.1 Likelihood ratio with source level propositions 245 5.3.2 Likelihood ratio with activity level propositions 255 5.3.3 Likelihood ratio with offence level propositions 271 5.4 Validation of Bayesian network structures: an example 275 5.5 Pre-assessment 278 5.5.1 Pre-assessment of the case 278 5.5.2 Pre-assessment of evidence 281 5.5.3 Pre-assessment: a practical example 281 5.6 Combination of items of evidence 288 5.6.1 A difficulty in combining evidence: the problem of conjunction 289 5.6.2 Generic patterns of inference in combining evidence 291 6 Evidence and propositions: practice 299 6.1 Examples for evaluation given source level propositions 299 6.1.1 A note on the appropriate databases for evaluation given source level propositions 301 6.1.2 Two trace problem 304 6.1.3 Many samples 307 6.1.4 Multiple propositions 310 6.1.5 A note on biological traces 318 6.1.6 Additional considerations on source level propositions 326 6.2 Examples for evaluation given activity level propositions 339 6.2.1 A practical approach to fibres evaluation 340 6.2.2 A practical approach to glass evaluation 341 6.2.3 The assignment of probabilities for transfer events 345 6.2.4 The assignment of probabilities for background traces 355 6.2.5 Presence of material with non-corresponding features 358 6.2.6 Absence of evidence for activity level propositions 358 6.3 Examples for evaluation given offence level propositions 360 6.3.1 One stain, k offenders 360 6.3.2 Two stains, one offender 364 6.3.3 Paternity and the combination of likelihood ratios 366 6.3.4 Probability of paternity 369 6.3.5 Absence of evidence for offence level propositions 372 6.3.6 A note on relevance and offence level propositions 374 6.4 Summary 375 6.4.1 Stain known to have been left by offenders: source-level propositions 375 6.4.2 Material known to have been (or not to have been) left by offenders: activity-level propositions 376 6.4.3 Stain may not have been left by offenders: offence level propositions 377 7 Data analysis 381 7.1 Introduction 381 7.2 Theory for discrete data 382 7.2.1 Data of independent counts with a Poisson distribution 383 7.2.2 Data of independent counts with a Binomial distribution 385 7.2.3 Data of independent counts with a multinomial distribution 386 7.3 Theory for continuous univariate data 388 7.3.1 Assessment of similarity only 390 7.3.2 Sources of variation: two-level models 394 7.3.3 Transfer probability 395 7.4 Normal between-source variation 397 7.4.1 Marginal distribution of measurements 397 7.4.2 Approximate derivation of the likelihood ratio 398 7.4.3 Lindley's approach 399 7.4.4 Interpretation of result 402 7.4.5 Examples 403 7.5 Non-normal between-source variation 405 7.5.1 Estimation of a probability density function 405 7.5.2 Kernel density estimation for between-source data 410 7.5.3 Examples 413 7.6 Multivariate analysis 416 7.6.1 Introduction 416 7.6.2 Multivariate two level models 417 7.6.3 A note on sensitivity 424 7.6.4 Case study for two-level data 424 7.6.5 Three-level models 429 7.7 Discrimination 433 7.7.1 Discrete data 434 7.7.2 Continuous data 436 7.7.3 Autocorrelated data 438 7.7.4 Multivariate data 439 7.7.5 Cut-offs and legal thresholds 441 7.8 Score-based models 444 7.8.1 Example 446 7.9 Bayes' factor and likelihood ratio (cont.) 448 8 Assessment of the performance of methods for the evaluation of evidence 451 8.1 Introduction 451 8.2 Properties of methods for evaluation 454 8.3 General topics relating to sample size estimation and to assessment 457 8.3.1 Probability of strong misleading evidence - a sample size problem 457 8.3.2 Calibration 465 8.4 Assessment of performance of a procedure for the calculation of the likelihood ratio 467 8.4.1 Histograms and Tippett plots 469 8.4.2 False positive rates, false negative rates and DET plots 470 8.4.3 Empirical cross-entropy 471 8.5 Case study: Kinship analysis 477 8.6 Conclusion 479 A Probability distributions 487 A.1 Introduction 487 A.2 Probability distributions for counts 490 A.2.1 Probabilities 490 A.2.2 Summary measures 491 A.2.3 Binomial distribution 494 A.2.4 Multinomial distribution 495 A.2.5 Hypergeometric distribution 495 A.2.6 Poisson distribution 496 A.2.7 Beta-binomial and Dirichlet-multinomial distributions 497 A.3 Measurements 499 A.3.1 Summary statistics 499 A.3.2 Normal distribution 500 A.3.3 Jeffreys' prior distributions 507 A.3.4 Student's t-distribution. 508 A.3.5 Gamma and chi-squared distributions 510 A.3.6 Inverse gamma and inverse chi-squared distributions 510 A.3.7 Beta distribution 511 A.3.8 Dirichlet distribution 513 A.3.9 Multivariate Normal distribution and correlation 515 A.3.10 Wishart distribution 517 A.3.11 Inverse Wishart distribution 518 B Matrix properties 519 B.1 Matrix terminology 519 B.1.1 The trace of a square matrix 519 B.1.2 The transpose of a matrix 519 B.1.3 Addition of two matrices 520 B.1.4 Determinant of a matrix 520 B.1.5 Matrix multiplication 521 B.1.6 The inverse of a matrix 522 B.1.7 Completion of the square 523 Notation 569 Cases 575 Author index 578 Subject index 593