Statistics for Scientists and Engineers

Preface xv About The Companion Website xxi 1 Descriptive Statistics 1 1.1 Introduction 1 1.2 Statistics as A Scientific Discipline 3 1.2.1 Scales of Measurement 4 1.3 The NOIR Scale 6 1.3.1 The Nominal Scale 6 1.3.2 The Ordinal Scale 6 1.3.3 The Interval Scale 7 1.3.4 The Ratio Scale 7 1.4 Population Versus Sample 8 1.4.1 Parameter Versus Statistic 9 1.5 Combination Notation 11 1.6 Summation Notation 11 1.6.1 Nested Sums 12 1.6.2 Increment Step Sizes 14 1.7 Product Notation 19 1.7.1 Evaluating Large Powers 20 1.8 Rising and Falling Factorials 22 1.9 Moments and Cumulants 22 1.10 Data Transformations 23 1.10.1 Change of Origin 23 1.10.2 Change of Scale 24 1.10.3 Change of Origin and Scale 24 1.10.4 Min-Max Transformation 25 1.10.5 Nonlinear Transformations 27 1.10.6 Standard Normalization 27 1.11 Data Discretization 28 1.12 Categorization of Data Discretization 28 1.12.1 Equal Interval Binning (EIB) 28 1.12.2 Equal Frequency Binning (EFB) 30 1.12.3 Entropy-Based Discretization (EBD) 31 1.12.4 Error in Discretization 34 1.13 Testing for Normality 34 1.13.1 Graphical Methods for Normality Checking 34 1.13.2 Ogive Plots 35 1.13.3 P-P and Q-Q Plots 35 1.13.4 Stem-and-Leaf Plots 36 1.13.5 Numerical Methods for Normality Testing 37 1.14 Summary 38 2 Measures of Location 43 2.1 Meaning of Location Measure 43 2.1.1 Categorization of Location Measures 44 2.2 Measures of Central Tendency 44 2.3 Arithmetic Mean 45 2.3.1 Updating Formula For Sample Mean 46 2.3.2 Sample Mean Using Change of Origin and Scale 49 2.3.3 Trimmed Mean 50 2.3.4 Weighted Mean 50 2.3.5 Mean of Grouped Data 51 2.3.6 Updating Formula for Weighted Sample Mean 51 2.3.7 Advantages of Mean 53 2.3.8 Properties of The Mean 53 2.4 Median 54 2.4.1 Median of Grouped Data 55 2.5 Quartiles and Percentiles 57 2.6 MODE 58 2.6.1 Advantages of Mode 58 2.7 Geometric Mean 59 2.7.1 Updating Formula For Geometric Mean 60 2.8 Harmonic Mean 61 2.8.1 Updating Formula For Harmonic Mean 61 2.9 Which Measure to Use? 63 2.10 Summary 63 3 Measures of Spread 67 3.1 Need For a Spread Measure 67 3.1.1 Categorization of Dispersion Measures 69 3.2 RANGE 71 3.2.1 Advantages of Range 72 3.2.2 Disadvantage of Range 72 3.2.3 Applications of Range 73 3.3 Inter-Quartile Range (IQR) 73 3.3.1 Change of Origin and Scale Transformation for Range 74 3.4 The Concept of Degrees of Freedom 74 3.5 Averaged Absolute Deviation (AAD) 76 3.5.1 Advantages of Averaged Absolute Deviation 77 3.5.2 Disadvantages of Averaged Absolute Deviation 77 3.5.3 Change of Origin and Scale Transformation for AAD 77 3.6 Variance and Standard Deviation 77 3.6.1 Advantages of Variance 78 3.6.2 Change of Origin and Scale Transformation for Variance 81 3.6.3 Disadvantages of Variance 81 3.6.4 A Bound for Sample Standard Deviation 82 3.7 Coefficient of Variation 82 3.7.1 Advantages of Coefficient of Variation 82 3.7.2 Disadvantages of Coefficient of Variation 83 3.7.3 An Interpretation of Coefficient of Variation 83 3.7.4 Change of Origin and Scale for CV 83 3.8 Gini Coefficient 84 3.9 Summary 84 4 Skewness and Kurtosis 89 4.1 Meaning of Skewness 89 4.1.1 Absolute Versus Relative Measures of Skewness 91 4.2 Categorization of Skewness Measures 93 4.3 Measures of Skewness 94 4.3.1 Bowley's Skewness Measure 94 4.3.2 Pearson's Skewness Measure 97 4.3.3 Coefficient of Quartile Deviation 98 4.3.4 Other Skewness Measures 99 4.4 Concept of Kurtosis 99 4.4.1 An Interpretation of Kurtosis 99 4.4.2 Categorization of Kurtosis Measures 101 4.5 Measures of Kurtosis 102 4.5.1 Pearson's Kurtosis Measure 102 4.5.2 Skewness-Kurtosis Bounds 104 4.5.3 L-kurtosis 104 4.5.4 Spectral Kurtosis (SK) 105 4.5.5 Detecting Faults Using SK 106 4.5.6 Multivariate Kurtosis 106 4.6 Summary 107 5 Probability 111 5.1 Introduction 111 5.2 Probability 112 5.3 Different Ways to Express Probability 114 5.3.1 Converting Nonrepeating Decimals to Fractions 114 5.3.2 Converting Repeating Decimals to Fractions 115 5.3.3 Converting Tail-Repeating Decimals to Fractions 116 5.4 Sample Space 119 5.5 Mathematical Background 121 5.5.1 Sets and Mappings 122 5.5.2 Venn Diagrams 124 5.5.3 Tree Diagrams 125 5.5.4 Bipartite Graphs 126 5.5.5 Bipartite Forests 126 5.6 Events 127 5.6.1 Deterministic and Probabilistic Events 128 5.6.2 Discrete Versus Continuous Events 128 5.6.3 Event Categories 129 5.6.4 Do-Little Principle for Events 131 5.7 Event Algebra 132 5.7.1 Laws of Events 132 5.7.2 De'Morgan's Laws 135 5.8 Basic Counting Principles 135 5.8.1 Rule of Sums (ROS) 135 5.8.2 Principle of Counting (POC) 136 5.8.3 Complete Enumeration 138 5.9 Permutations and Combinations 140 5.9.1 Permutations with Restrictions 142 5.9.2 Permutation of Alike Objects 142 5.9.3 Cyclic Permutations 143 5.9.4 Cyclic Permutations of Subsets 144 5.9.5 Combinations 145 5.10 Principle of Inclusion and Exclusion (PIE) 147 5.11 Recurrence Relations 149 5.11.1 Derangements and Matching Problems 149 5.12 Urn Models 152 5.13 Partitions 154 5.14 Axiomatic Approach 154 5.14.1 Probability Measure 155 5.14.2 Probability Space 155 5.15 The Classical Approach 156 5.15.1 Counting Techniques in Classical Probability 156 5.15.2 Assigning Probabilities to Events 156 5.15.3 Rules of Probability 157 5.15.4 Do-Little Principle of Probability 159 5.15.5 Permutation and Combination in Classical Approach 161 5.15.6 Sequentially Dependent Events 163 5.15.7 Independence of Events 163 5.15.8 Independent Random Variables 165 5.16 Frequency Approach 166 5.16.1 Entropy Versus Probability 168 5.17 Bayes Theorem 168 5.17.1 Bayes Theorem for Conditional Probability 169 5.17.2 Bayes Classification Rule 172 5.18 Summary 173 6 Discrete Distributions 185 6.1 Discrete Random Variables 185 6.2 Binomial Theorem 186 6.2.1 Recurrence Relation for Binomial Coefficients 187 6.2.2 Distributions Obtainable from Binomial Theorem 188 6.3 Mean Deviation of Discrete Distributions 189 6.3.1 Recurrence Relation for Mean Deviation 192 6.4 Bernoulli Distribution 192 6.5 Binomial Distribution 194 6.5.1 Properties of Binomial Distribution 195 6.5.2 Moment Recurrences 197 6.5.3 Additivity Property 202 6.5.4 Distribution of the Difference of Successes and Failures 203 6.5.5 Algorithm for Binomial Distribution 206 6.5.6 Tail Probabilities 207 6.5.7 Approximations 209 6.5.8 Limiting Form of Binomial Distribution 209 6.6 Discrete Uniform Distribution 211 6.6.1 Properties of Discrete Uniform Distribution 212 6.6.2 An Application 214 6.7 Geometric Distribution 214 6.7.1 Properties of Geometric Distribution 219 6.7.2 Memory-less Property 221 6.7.3 Tail Probabilities 222 6.7.4 Random Samples 222 6.8 Negative Binomial Distribution 223 6.8.1 Properties of Negative Binomial Distribution 223 6.8.2 Moment Recurrence 227 6.8.3 Tail Probabilities 228 6.9 Poisson Distribution 229 6.9.1 Properties of Poisson Distribution 230 6.9.2 Algorithms for Poisson Distribution 236 6.9.3 Truncated Poisson Distribution 237 6.10 Hypergeometric Distribution 238 6.10.1 Properties of Hypergeometric Distribution 238 6.10.2 Moments of Hypergeometric Distribution 239 6.10.3 Approximations for Hypergeometric Distribution 241 6.11 Negative Hypergeometric Distribution 241 6.12 Beta Binomial Distribution 241 6.13 Logarithmic Series Distribution 242 6.13.1 Properties of Logarithmic Distribution 243 6.14 Multinomial Distribution 243 6.14.1 Properties of Multinomial Distribution 244 6.15 Summary 246 7 Continuous Distributions 255 7.1 Introduction 255 7.2 Mean Deviation of Continuous Distributions 256 7.2.1 Notion of Infinity 260 7.3 Continuous Uniform Distribution 260 7.3.1 Properties of Continuous Uniform Distribution 261 7.3.2 Relationships with Other Distributions 264 7.3.3 Applications 264 7.4 Exponential Distribution 265 7.4.1 Properties of Exponential Distribution 265 7.4.2 Additivity Property 266 7.5 Beta Distribution 269 7.5.1 Type-I Beta Distribution 269 7.5.2 Properties of Type-I Beta Distribution 270 7.5.3 Type-II Beta Distribution 274 7.5.4 Properties of Type-II Beta Distribution 274 7.5.5 Relationship with Other Distributions 276 7.6 The Incomplete Beta Function 276 7.6.1 Tail Areas Using IBF 277 7.6.2 Tables 278 7.7 General Beta Distribution 278 7.8 Arc-Sine Distribution 279 7.8.1 Properties of Arc-Sine Distribution 281 7.9 Gamma Distribution 282 7.9.1 Properties of Gamma Distribution 283 7.9.2 Relationships with Other Distributions 284 7.9.3 Incomplete Gamma Function (IGF) 285 7.10 Cosine Distribution 285 7.11 The Normal Distribution 286 7.11.1 Properties of Normal Distribution 287 7.11.2 Transformations to Normality 290 7.11.3 Functions of Normal Variates 290 7.11.4 Relation to Other Distributions 291 7.11.5 Algorithms 293 7.12 Cauchy Distribution 293 7.12.1 Properties of Cauchy Distribution 294 7.12.2 Functions of Cauchy Variate 294 7.12.3 Relation to Other Distributions 295 7.13 Inverse Gaussian Distribution 295 7.13.1 Relation to Other Distributions 296 7.14 Lognormal Distribution 296 7.14.1 Properties of Lognormal Distribution 298 7.14.2 Moments 299 7.14.3 Fitting Lognormal Distribution 302 7.15 Pareto Distribution 302 7.15.1 Properties of Pareto Distribution 302 7.15.2 Relation to Other Distributions 303 7.15.3 Algorithms 303 7.16 Double Exponential Distribution 304 7.16.1 Relation to Other Distributions 307 7.17 Central 2 Distribution 307 7.17.1 Properties of Central 2 Distribution 308 7.17.2 Relationships with Other Distributions 310 7.18 Student's T Distribution 310 7.18.1 Properties of Student's T Distribution 310 7.18.2 Relation to Other Distributions 313 7.19 Snedecor's F Distribution 315 7.19.1 Properties of F Distribution 315 7.19.2 Relation to Other Distributions 316 7.20 Fisher's Z Distribution 317 7.20.1 Properties of Fisher's Z Distribution 317 7.21 Weibull Distribution 319 7.21.1 Properties of Weibull Distribution 319 7.21.2 Random Numbers 321 7.22 Rayleigh Distribution 321 7.22.1 Properties of Rayleigh Distribution 323 7.23 Chi-Distribution 323 7.23.1 Properties of Chi-Distribution 324 7.24 Maxwell Distribution 324 7.24.1 Properties of Maxwell Distribution 324 7.25 Summary 326 8 Mathematical Expectation 333 8.1 Meaning of Expectation 333 8.2 Random Variable 334 8.2.1 Cumulative Distribution Function 335 8.2.2 Expected Value 337 8.2.3 Range for Summation or Integration 341 8.2.4 Expectation Using Distribution Functions 342 8.3 Expectation of Functions of Random Variables 346 8.3.1 Properties of Expectations 346 8.3.2 Expectation of Continuous Functions 350 8.3.3 Variance as Expected Value 352 8.3.4 Covariance as Expected Value 353 8.3.5 Moments as Expected Values 355 8.4 Conditional Expectations 355 8.4.1 Conditional Variances 359 8.4.2 Law of Conditional Variances 360 8.5 Inverse Moments 361 8.6 Incomplete Moments 362 8.7 Distances as Expected Values 362 8.7.1 Chebychev Inequality 362 8.8 Summary 363 9 Generating Functions 373 9.1 Types of Generating Functions 373 9.1.1 Generating Functions in Statistics 375 9.2 Probability Generating Functions (PGF) 375 9.2.1 Properties of PGF 377 9.3 Generating Functions for CDF (GFCDF) 378 9.4 Generating Functions for Mean Deviation (GFMD) 379 9.5 Moment Generating Functions (MGF) 380 9.5.1 Properties of Moment Generating Functions 381 9.6 Characteristic Functions (ChF) 384 9.6.1 Properties of Characteristic Functions 385 9.7 Cumulant Generating Functions (CGF) 387 9.7.1 Relations Among Moments and Cumulants 387 9.8 Factorial Moment Generating Functions (FMGF) 389 9.9 Conditional Moment Generating Functions (CMGF) 390 9.10 Convergence of Generating Functions 391 9.11 Summary 391 10 Functions of Random Variables 395 10.1 Functions of Random Variables 395 10.2 Distribution of Translations 397 10.3 Distribution of Constant Multiples 397 10.4 Method of Distribution Functions (MoDF) 398 10.4.1 Distribution of Absolute Value (|X|) Using MoDF 399 10.4.2 Distribution of F(x) and F.1(x) Using MoDF 399 10.5 Change of Variable Technique 401 10.5.1 Linear Transformations 402 10.6 Distribution of Squares 403 10.7 Distribution of Square-Roots 404 10.8 Distribution of Reciprocals 406 10.9 Distribution of Minimum and Maximum 406 10.10 Distribution of Trigonometric Functions 407 10.11 Distribution of Transcendental Functions 407 10.11.1 Distribution of Sums 408 10.11.2 Distribution of Arbitrary Functions 410 10.11.3 Distribution of Logarithms 412 10.11.4 Special Functions 412 10.12 Transformations of Normal Variates 413 10.12.1 Linear Combination of Normal Variates 413 10.12.2 Square of Normal Variates 413 10.13 Summary 414 11 Joint Distributions 417 11.1 Joint and Conditional Distributions 417 11.1.1 Marginal Distributions 418 11.1.2 Conditional Distributions 419 11.2 Jacobian of Transformations 421 11.2.1 Functions of Several Variables 422 11.2.2 Arbitrary Transformations 422 11.2.3 Image Jacobian Matrices 425 11.2.4 Distribution of Products and Ratios 427 11.3 Polar Transformations 433 11.3.1 Plane Polar Transformations (PPT) 433 11.3.2 Cylindrical Polar Transformations (CPT) 436 11.3.3 Spherical Polar Transformations (SPT) 436 11.3.4 Other Methods 436 11.4 Summary 438 References 441 Index 455