Mathematical Statistics
Description
This book introduces readers to point estimation, confidence intervals, and statistical tests. Based on the general theory of linear models, it provides an in-depth overview of the following: analysis of variance (ANOVA) for models with fixed, random, and mixed effects; regression analysis is also first presented for linear models with fixed, random, and mixed effects before being expanded to nonlinear models; statistical multi-decision problems like statistical selection procedures (Bechhofer and Gupta) and sequential tests; and design of experiments from a mathematical-statistical point of view. Most analysis methods have been supplemented by formulae for minimal sample sizes. The chapters also contain exercises with hints for solutions.
Translated from the successful German text, Mathematical Statistics requires knowledge of probability theory (combinatorics, probability distributions, functions and sequences of random variables), which is typically taught in the earlier semesters of scientific and mathematical study courses. It teaches readers all about statistical analysis and covers the design of experiments. The book also describes optimal allocation in the chapters on regression analysis. Additionally, it features a chapter devoted solely to experimental designs.
* Classroom-tested with exercises included
* Practice-oriented (taken from day-to-day statistical work of the authors)
* Includes further studies including design of experiments and sample sizing
* Presents and uses IBM SPSS Statistics 24 for practical calculations of data
Mathematical Statistics is a recommended text for advanced students and practitioners of math, probability, and statistics.
More details
Other editions
Additional editions
Persons
DIETER RASCH, PhD, is scientific advisor at the Center for Design of Experiments at the University of Natural Resources and Life Sciences, Vienna, Austria. He has published more than 275 scientific papers and 56 books as author or editor.
DIETER SCHOTT obtained his PhD in analysis from the University of Rostock in 1976 and did his habilitation in the field of numerical functional analysis in 1982. He has published more than 100 scientific papers and is active as author, co-author and editor of numerous books and scientific journals.
Content
Preface xiii
1 Basic Ideas of Mathematical Statistics 1
1.1 Statistical Population and Samples 2
1.1.1 Concrete Samples and Statistical Populations 2
1.1.2 Sampling Procedures 4
1.2 Mathematical Models for Population and Sample 8
1.3 Sufficiency and Completeness 9
1.4 The Notion of Information in Statistics 20
1.5 Statistical Decision Theory 28
1.6 Exercises 32
References 37
2 Point Estimation 39
2.1 Optimal Unbiased Estimators 41
2.2 Variance-Invariant Estimation 53
2.3 Methods for Construction and Improvement of Estimators 57
2.3.1 Maximum Likelihood Method 57
2.3.2 Least Squares Method 60
2.3.3 Minimum Chi-Squared Method 61
2.3.4 Method of Moments 62
2.3.5 Jackknife Estimators 63
2.3.6 Estimators Based on Order Statistics 64
2.3.6.1 Order and Rank Statistics 64
2.3.6.2 L-Estimators 66
2.3.6.3 M-Estimators 67
2.3.6.4 R-Estimators 68
2.4 Properties of Estimators 68
2.4.1 Small Samples 69
2.4.2 Asymptotic Properties 71
2.5 Exercises 75
References 78
3 Statistical Tests and Confidence Estimations 79
3.1 Basic Ideas of Test Theory 79
3.2 The Neyman-Pearson Lemma 87
3.3 Tests for Composite Alternative Hypotheses and One-Parametric Distribution Families 96
3.3.1 Distributions with Monotone Likelihood Ratio and Uniformly Most Powerful Tests for One-Sided Hypotheses 96
3.3.2 UMPU-Tests for Two-Sided Alternative Hypotheses 105
3.4 Tests for Multi-Parametric Distribution Families 110
3.4.1 General Theory 111
3.4.2 The Two-Sample Problem: Properties of Various Tests and Robustness 124
3.4.2.1 Comparison of Two Expectations 125
3.4.3 Comparison of Two Variances 137
3.4.4 Table for Sample Sizes 138
3.5 Confidence Estimation 139
3.5.1 One-Sided Confidence Intervals in One-Parametric Distribution Families 140
3.5.2 Two-Sided Confidence Intervals in One-Parametric and Confidence Intervals in Multi-Parametric Distribution Families 143
3.5.3 Table for Sample Sizes 146
3.6 Sequential Tests 147
3.6.1 Introduction 147
3.6.2 Wald's Sequential Likelihood Ratio Test for One-Parametric Exponential Families 149
3.6.3 Test about Mean Values for Unknown Variances 153
3.6.4 Approximate Tests for the Two-Sample Problem 158
3.6.5 Sequential Triangular Tests 160
3.6.6 A Sequential Triangular Test for the Correlation Coefficient 162
3.7 Remarks about Interpretation 169
3.8 Exercises 170
References 176
4 Linear Models - General Theory 179
4.1 Linear Models with Fixed Effects 179
4.1.1 Least Squares Method 180
4.1.2 Maximum Likelihood Method 184
4.1.3 Tests of Hypotheses 185
4.1.4 Construction of Confidence Regions 190
4.1.5 Special Linear Models 191
4.1.6 The Generalised Least Squares Method (GLSM) 198
4.2 Linear Models with Random Effects: Mixed Models 199
4.2.1 Best Linear Unbiased Prediction (BLUP) 200
4.2.2 Estimation of Variance Components 202
4.3 Exercises 203
References 204
5 Analysis of Variance (ANOVA) - Fixed Effects Models (Model I of Analysis of Variance) 207
5.1 Introduction 207
5.2 Analysis of Variance with One Factor (Simple- or One-Way Analysis of Variance) 215
5.2.1 The Model and the Analysis 215
5.2.2 Planning the Size of an Experiment 228
5.2.2.1 General Description for All Sections of This Chapter 228
5.2.2.2 The Experimental Size for the One-Way Classification 231
5.3 Two-Way Analysis of Variance 232
5.3.1 Cross-Classification (A × B) 233
5.3.1.1 Parameter Estimation 236
5.3.1.2 Testing Hypotheses 244
5.3.2 Nested Classification (A B) 260
5.4 Three-Way Classification 272
5.4.1 Complete Cross-Classification (A × B × C) 272
5.4.2 Nested Classification (C ¿B¿A) 279
5.4.3 Mixed Classification 282
5.4.3.1 Cross-Classification between Two Factors Where One of Them Is Subordinated to a Third Factor B¿a × c 282
5.4.3.2 Cross-Classification of Two Factors in Which a Third Factor Is Nested C ¿ A × B 288
5.5 Exercises 291
References 291
6 Analysis of Variance: Estimation of Variance Components (Model II of the Analysis of Variance) 293
6.1 Introduction: Linear Models with Random Effects 293
6.2 One-Way Classification 297
6.2.1 Estimation of Variance Components 300
6.2.1.1 Analysis of Variance Method 300
6.2.1.2 Estimators in Case of Normally Distributed Y 302
6.2.1.3 REML Estimation 304
6.2.1.4 Matrix Norm Minimising Quadratic Estimation 305
6.2.1.5 Comparison of Several Estimators 306
6.2.2 Tests of Hypotheses and Confidence Intervals 308
6.2.3 Variances and Properties of the Estimators of the Variance Components 310
6.3 Estimators of Variance Components in the Two-Way and Three-Way Classification 315
6.3.1 General Description for Equal and Unequal Subclass Numbers 315
6.3.2 Two-Way Cross-Classification 319
6.3.3 Two-Way Nested Classification 324
6.3.4 Three-Way Cross-Classification with Equal Subclass Numbers 326
6.3.5 Three-Way Nested Classification 333
6.3.6 Three-Way Mixed Classification 335
6.4 Planning Experiments 336
6.5 Exercises 338
References 339
7 Analysis of Variance - Models with Finite Level Populations and Mixed Models 341
7.1 Introduction: Models with Finite Level Populations 341
7.2 Rules for the Derivation of SS, df, MS and E(MS) in Balanced ANOVA Models 343
7.3 Variance Component Estimators in Mixed Models 348
7.3.1 An Example for the Balanced Case 349
7.3.2 The Unbalanced Case 351
7.4 Tests for Fixed Effects and Variance Components 353
7.5 Variance Component Estimation and Tests of Hypotheses in Special Mixed Models 354
7.5.1 Two-Way Cross-Classification 355
7.5.2 Two-Way Nested Classification B ¿ A 358
7.5.2.1 Levels of A Random 360
7.5.2.2 Levels of B Random 361
7.5.3 Three-Way Cross-Classification 362
7.5.4 Three-Way Nested Classification 365
7.5.5 Three-Way Mixed Classification 368
7.5.5.1 The Type (B ¿ A)×C 368
7.5.5.2 The Type C ¿ AB 371
7.6 Exercises 374
References 374
8 Regression Analysis - Linear Models with Non-random Regressors (Model I of Regression Analysis) and with Random Regressors (Model II of Regression Analysis) 377
8.1 Introduction 377
8.2 Parameter Estimation 380
8.2.1 Least Squares Method 380
8.2.2 Optimal Experimental Design 394
8.3 Testing Hypotheses 397
8.4 Confidence Regions 406
8.5 Models with Random Regressors 410
8.5.1 Analysis 410
8.5.2 Experimental Designs 415
8.6 Mixed Models 416
8.7 Concluding Remarks about Models of Regression Analysis 417
8.8 Exercises 419
References 419
9 Regression Analysis - Intrinsically Non-linear Model I 421
9.1 Estimating by the Least Squares Method 424
9.1.1 Gauss-Newton Method 425
9.1.2 Internal Regression 431
9.1.3 Determining Initial Values for Iteration Methods 433
9.2 Geometrical Properties 434
9.2.1 Expectation Surface and Tangent Plane 434
9.2.2 Curvature Measures 440
9.3 Asymptotic Properties and the Bias of LS Estimators 443
9.4 Confidence Estimations and Tests 447
9.4.1 Introduction 447
9.4.2 Tests and Confidence Estimations Based on the Asymptotic Covariance Matrix 451
9.4.3 Simulation Experiments to Check Asymptotic Tests and Confidence Estimations 452
9.5 Optimal Experimental Design 454
9.6 Special Regression Functions 458
9.6.1 Exponential Regression 458
9.6.1.1 Point Estimator 458
9.6.1.2 Confidence Estimations and Tests 460
9.6.1.3 Results of Simulation Experiments 463
9.6.1.4 Experimental Designs 466
9.6.2 The Bertalanffy Function 468
9.6.3 The Logistic (Three-Parametric Hyperbolic Tangent) Function 473
9.6.4 The Gompertz Function 476
9.6.5 The Hyperbolic Tangent Function with Four Parameters 480
9.6.6 The Arc Tangent Function with Four Parameters 484
9.6.7 The Richards Function 487
9.6.8 Summarising the Results of Sections 9.6.1-9.6.7 487
9.6.9 Problems of Model Choice 488
9.7 Exercises 489
References 490
10 Analysis of Covariance (ANCOVA) 495
10.1 Introduction 495
10.2 General Model I-I of the Analysis of Covariance 496
10.3 Special Models of the Analysis of Covariance for the Simple Classification 503
10.3.1 One Covariable with Constant ¿ 504
10.3.2 A Covariable with Regression Coefficients G I Depending on the Levels of the Classification Factor 506
10.3.3 A Numerical Example 507
10.4 Exercises 510
References 511
11 Multiple Decision Problems 513
11.1 Selection Procedures 514
11.1.1 Basic Ideas 514
11.1.2 Indifference Zone Formulation for Expectations 516
11.1.2.1 Selection of Populations with Normal Distribution 517
11.1.2.2 Approximate Solutions for Non-normal Distributions and t =1 529
11.1.3 Selection of a Subset Containing the Best Population with Given Probability 531
11.1.3.1 Selection of the Normal Distribution with the Largest Expectation 534
11.1.3.2 Selection of the Normal Distribution with Smallest Variance 534
11.2 Multiple Comparisons 539
11.2.1 Confidence Intervals for All Contrasts: Scheffé's Method 542
11.2.2 Confidence Intervals for Given Contrasts: Bonferroni's and Dunn's Method 548
11.2.3 Confidence Intervals for All Contrasts for N I = N: Tukey's Method 550
11.2.4 Confidence Intervals for All Contrasts: Generalised Tukey's Method 553
11.2.5 Confidence Intervals for the Differences of Treatments with a Control: Dunnett's Method 554
11.2.6 Multiple Comparisons and Confidence Intervals 556
11.2.7 Which Multiple Comparison Shall Be Used? 559
11.3 A Numerical Example 559
11.4 Exercises 563
References 563
12 Experimental Designs 567
12.1 Introduction 568
12.2 Block Designs 571
12.2.1 Completely Balanced Incomplete Block Designs (BIBD) 574
12.2.2 Construction Methods of BIBD 582
12.2.3 Partially Balanced Incomplete Block Designs 596
12.3 Row-Column Designs 600
12.4 Factorial Designs 603
12.5 Programs for Construction of Experimental Designs 604
12.6 Exercises 604
References 605
Appendix A: Symbolism 609
Appendix B: Abbreviations 611
Appendix C: Probability and Density Functions 613
Appendix D: Tables 615
Solutions and Hints for Exercises 627
Index 659
1
Basic Ideas of Mathematical Statistics
Elementary statistical computations have been carried out for thousands of years. For example, the arithmetic mean from a number of measures or observation data has been known for a very long time.
First descriptive statistics arose starting with the collection of data, for example, at the national census or in registers of medical cards, and followed by compression of these data in the form of statistics or graphical representations (figures). Mathematical statistics developed on the fundament of probability theory from the end of 19th century on. At the beginning of the 20th century, Karl Pearson and Sir Ronald Aylmer Fisher were notable pioneers of this new discipline. Fisher's book (1925) was a milestone providing experimenters such basic concepts as his well-known maximum likelihood method and analysis of variance as well as notions of sufficiency and efficiency. An important information measure is still called the Fisher information (see Section 1.4).
Concerning historical development we do not want to go into detail. We refer interested readers to Stigler (1986, 1990). Instead we will describe the actual state of the theory. Nevertheless many stimuli come from real applications. Hence, from time to time we will include real examples.
Although the probability calculus is the fundament of mathematical statistics, many practical problems containing statements about random variables cannot be solved with this calculus alone. For example, we often look for statements about parameters of distribution functions although we do not partly or completely know these functions. Mathematical statistics is considered in many introductory textbooks as the theory of analysing experiments or samples; that is, it is assumed that a random sample (corresponding to Section 1.1) is given. Often it is not considered how to get such a random sample in an optimal way. This is treated later in design of experiments. But in concrete applications, the experiment first has to be planned, and after the experiment is finished, the analysis has to be carried out. But in theory it is appropriate to determine firstly the optimal evaluation, for example, the smallest sample size for a variance optimal estimator. Hence we proceed in such a way and start with the optimal evaluation, and after this we work out the design problems. An exception is made for sequential methods where planning and evaluation are realised together.
Mathematical statistics involves mainly the theory of point estimation, statistical selection theory, the theory of hypothesis testing and the theory of confidence estimation. In these areas theorems are proved, showing which procedures are the best ones under special assumptions.
We wish to make clear that the treatment of mathematical statistics on the one hand and its application to concrete data material on the other hand are totally different concepts. Although the same terms often occur, they need not be confused. Strictly speaking, the notions of the empirical sphere (hence of the real world) are related to corresponding models in theory.
If assumptions for deriving best methods are not fulfilled in practical applications, the question arises how good these best methods still are. Such questions are answered by a part of empirical statistics - by simulations. We often find that the assumption of a normal distribution occurring in many theorems is far from being a good model for many data in applications. In the last years simulation developed into its own branch in mathematics. This shows a series of international workshops on simulation. The first to sixth workshops took place in St. Petersburg (Russia) in 1994, 1996, 1998, 2001, 2005 and 2009. The seventh international workshop on simulation took place in Rimini (Italy) in 2013 and the eighth one in Vienna (Austria) in 2015.
Because the strength of assumptions has consequences mainly in hypothesis testing and confidence estimation, we discuss such problems first in Chapter 3, where we introduce the concept of robustness against the strength of assumptions.
1.1 Statistical Population and Samples
1.1.1 Concrete Samples and Statistical Populations
In the empirical sciences, one character or several characters simultaneously (character vector) are observed in certain objects (or individuals) of a population. The main task is to conclude from the sample of observed values to the whole set of character values of all objects of this population. The problem is that there are objective or economical points of view that do not admit the complete survey of all character values in the population. We give some examples:
- The costs to register all character values were out of all proportion to the value of the statement (for instance, measuring the height of all people worldwide older than 18 years).
- The registration of character values results in destruction of the objects (destructive materials testing such as resistance to tearing of ropes or stockings).
- The set of objects is of hypothetic nature, for example, because they partly do not exist at the moment of investigation (as all products of a machine).
We can neglect the few practical cases where all objects of a population can be observed and no more extensive population is demanded, because for them mathematical statistics is not needed. Therefore we assume that a certain part (subset) is chosen from the population to observe a character (or character vector) from which we want to draw conclusions to the whole population. We call such a part a (concrete) sample (of the objects). The set of character values measured for these objects is said to be a (concrete) sample of the character values. Each object of the population is to possess such a character value (independent of whether we register the value or not). The set of character values of all objects in the population is called the corresponding statistical population.
A concrete population as well as the (sought-after/relevant) character and therefore also the corresponding statistical population need to be determined uniquely. Populations have to be circumscribed in the first line in relation to space and time. In principle it must be clear for an arbitrary real object whether it belongs to the population or not. In the following we consider some examples:
Original population Statistical population A Heifer of a certain breed in a certain region in a certain year A1 Yearly yield of milk of these heifer A2 Body mass of these heifer after 180 days A3 Back height of these heifer B Inhabitants of a town at a certain day B1 Blood pressure of these inhabitants at 6.00 o'clock B2 Age of these inhabitantsIt is clear that applying conclusions from a sample to the whole population can be wrong. For example, if the children of a day nursery are chosen from the population B in the table above, then possibly the blood pressure B1 but without doubt the age B2 are not applicable to B. Generally we speak of characters, but if they can have a certain influence to the experimental results, they are also called factors. The (mostly only a few) character values are said to be factor levels, and the combinations of factor levels of several factors factor level combinations.
The sample should be representative with respect to all factors that can influence the character of a statistical population. That means the composition of the population should be mirrored in the sample of objects. But that is impossible for small samples and many factor level combinations. For example, there are already about 200 factor level combinations in population B concerning the factors age and sex, which cannot be representatively found in a sample of 100 inhabitants. Therefore we recommend avoiding the notion of 'representative sample' because it cannot be defined in a correct way.
Samples should not be assessed according to the elements included but according to the way these elements have been selected. This way of selecting a sample is called sampling procedure. It can be applied either to the objects as statistical units or to the population of character values (e.g. in a databank). In the latter case the sample of character values arises immediately. In the first case the character must be first registered at the selected objects. Both procedures (but not necessarily the created samples) are equivalent if the character value is registered for each registered object. This is assumed in this chapter. It is not the case in so-called censored samples where the character values could not be registered in all units of the experiment. For example, if the determination of lifespans of objects (as electronic components) is finished at a certain time, measured values of objects with longer lifespans (as time of determination) are missing.
In the following we do not differ between samples of objects and samples of character values; the definitions hold for both.
Definition 1.1
A sampling procedure is a rule of selecting a proper subset, named sample, from a well-defined finite basic set of objects (population, universe). It is said to be at random if each element of the basic set has the...
