Handbook of Probability

List of Figures xv Preface xvii Introduction xix 1 Probability Space 1 1.1 Introduction/Purpose of the Chapter 1 1.2 Vignette/Historical Notes 2 1.3 Notations and Definitions 2 1.4 Theory and Applications 4 1.4.1 Algebras 4 1.4.2 Sigma Algebras 5 1.4.3 Measurable Spaces 7 1.4.4 Examples 7 1.4.5 The Borel sigma-Algebra 9 1.5 Summary 12 Exercises 12 2 Probability Measure 15 2.1 Introduction/Purpose of the Chapter 15 2.2 Vignette/Historical Notes 16 2.3 Theory and Applications 17 2.3.1 Definition and Basic Properties 17 2.3.2 Uniqueness of Probability Measures 22 2.3.3 Monotone Class 24 2.3.4 Examples 26 2.3.5 Monotone Convergence Properties of Probability 28 2.3.6 Conditional Probability 31 2.3.7 Independence of Events and sigma-Fields 39 2.3.8 Borel-Cantelli Lemmas 46 2.3.9 Fatou's Lemmas 48 2.3.10 Kolmogorov's Zero-One Law 49 2.4 Lebesgue Measure on the Unit Interval (01] 50 Exercises 52 3 Random Variables: Generalities 63 3.1 Introduction/Purpose of the Chapter 63 3.2 Vignette/Historical Notes 63 3.3 Theory and Applications 64 3.3.1 Definition 64 3.3.2 The Distribution of a Random Variable 65 3.3.3 The Cumulative Distribution Function of a Random Variable 67 3.3.4 Independence of Random Variables 70 Exercises 71 4 Random Variables: The Discrete Case 79 4.1 Introduction/Purpose of the Chapter 79 4.2 Vignette/Historical Notes 80 4.3 Theory and Applications 80 4.3.1 Definition and Basic Facts 80 4.3.2 Moments 84 4.4 Examples of Discrete Random Variables 89 4.4.1 The (Discrete) Uniform Distribution 89 4.4.2 Bernoulli Distribution 91 4.4.3 Binomial (n p) Distribution 92 4.4.4 Geometric (p) Distribution 95 4.4.5 Negative Binomial (r p) Distribution 101 4.4.6 Hypergeometric Distribution (N m n) 102 4.4.7 Poisson Distribution 104 Exercises 108 5 Random Variables: The Continuous Case 119 5.1 Introduction/Purpose of the Chapter 119 5.2 Vignette/Historical Notes 119 5.3 Theory and Applications 120 5.3.1 Probability Density Function (p.d.f.) 120 5.3.2 Cumulative Distribution Function (c.d.f.) 124 5.3.3 Moments 127 5.3.4 Distribution of a Function of the Random Variable 128 5.4 Examples 130 5.4.1 Uniform Distribution on an Interval [ab] 130 5.4.2 Exponential Distribution 133 5.4.3 Normal Distribution (my, sigma²) 136 5.4.4 Gamma Distribution 139 5.4.5 Beta Distribution 144 5.4.6 Student's t Distribution 147 5.4.7 Pareto Distribution 149 5.4.8 The Log-Normal Distribution 151 5.4.9 Laplace Distribution 153 5.4.10 Double Exponential Distribution 155 Exercises 156 6 Generating Random Variables 177 6.1 Introduction/Purpose of the Chapter 177 6.2 Vignette/Historical Notes 178 6.3 Theory and Applications 178 6.3.1 Generating One-Dimensional Random Variables by Inverting the Cumulative Distribution Function (c.d.f.) 178 6.3.2 Generating One-Dimensional Normal Random Variables 183 6.3.3 Generating Random Variables. Rejection Sampling Method 186 6.3.4 Generating from a Mixture of Distributions 193 6.3.5 Generating Random Variables. Importance Sampling 195 6.3.6 Applying Importance Sampling 198 6.3.7 Practical Consideration: Normalizing Distributions 201 6.3.8 Sampling Importance Resampling 203 6.3.9 Adaptive Importance Sampling 204 6.4 Generating Multivariate Distributions with Prescribed Covariance Structure 205 Exercises 208 7 Random Vectors in Rn 210 7.1 Introduction/Purpose of the Chapter 210 7.2 Vignette/Historical Notes 210 7.3 Theory and Applications 211 7.3.1 The Basics 211 7.3.2 Marginal Distributions 212 7.3.3 Discrete Random Vectors 214 7.3.4 Multinomial Distribution 219 7.3.5 Testing Whether Counts are Coming from a Specific Multinomial Distribution 220 7.3.6 Independence 221 7.3.7 Continuous Random Vectors 223 7.3.8 Change of Variables. Obtaining Densities of Functions of Random Vectors 229 7.3.9 Distribution of Sums of Random Variables. Convolutions 231 Exercises 236 8 Characteristic Function 255 8.1 Introduction/Purpose of the Chapter 255 8.2 Vignette/Historical Notes 255 8.3 Theory and Applications 256 8.3.1 Definition and Basic Properties 256 8.3.2 The Relationship Between the Characteristic Function and the Distribution 260 8.4 Calculation of the Characteristic Function for Commonly Encountered Distributions 265 8.4.1 Bernoulli and Binomial 265 8.4.2 Uniform Distribution 266 8.4.3 Normal Distribution 267 8.4.4 Poisson Distribution 267 8.4.5 Gamma Distribution 268 8.4.6 Cauchy Distribution 269 8.4.7 Laplace Distribution 270 8.4.8 Stable Distributions. L´evy Distribution 271 8.4.9 Truncated L´evy Flight Distribution 274 Exercises 275 9 Moment-Generating Function 280 9.1 Introduction/Purpose of the Chapter 280 9.2 Vignette/Historical Notes 280 9.3 Theory and Applications 281 9.3.1 Generating Functions and Applications 281 9.3.2 Moment-Generating Functions. Relation with the Characteristic Functions 288 9.3.3 Relationship with the Characteristic Function 292 9.3.4 Properties of the MGF 292 Exercises 294 10 Gaussian Random Vectors 300 10.1 Introduction/Purpose of the Chapter 300 10.2 Vignette/Historical Notes 301 10.3 Theory and Applications 301 10.3.1 The Basics 301 10.3.2 Equivalent Definitions of a Gaussian Vector 303 10.3.3 Uncorrelated Components and Independence 309 10.3.4 The Density of a Gaussian Vector 313 10.3.5 Cochran's Theorem 316 10.3.6 Matrix Diagonalization and Gaussian Vectors 319 Exercises 325 11 Convergence Types. Almost Sure Convergence. Lp-Convergence. Convergence in Probability 338 11.1 Introduction/Purpose of the Chapter 338 11.2 Vignette/Historical Notes 339 11.3 Theory and Applications: Types of Convergence 339 11.3.1 Traditional Deterministic Convergence Types 339 11.3.2 Convergence of Moments of an r.v.--Convergence in Lp 341 11.3.3 Almost Sure (a.s.) Convergence 342 11.3.4 Convergence in Probability 344 11.4 Relationships Between Types of Convergence 346 11.4.1 a.s. and Lp 347 11.4.2 Probability and a.s./Lp 351 11.4.3 Uniform Integrability 357 Exercises 359 12 Limit Theorems 372 12.1 Introduction/Purpose of the Chapter 372 12.2 Vignette/Historical Notes 372 12.3 Theory and Applications 375 12.3.1 Weak Convergence 375 12.3.2 The Law of Large Numbers 384 12.4 Central Limit Theorem 401 Exercises 409 13 Appendix A: Integration Theory. General Expectations 421 13.1 Integral of Measurable Functions 422 13.1.1 Integral of Simple (Elementary) Functions 422 13.1.2 Integral of Positive Measurable Functions 424 13.1.3 Integral of Measurable Functions 428 13.2 General Expectations and Moments of a Random Variable 429 13.2.1 Moments and Central Moments. Lp Space 430 13.2.2 Variance and the Correlation Coefficient 431 13.2.3 Convergence Theorems 433 14 Appendix B: Inequalities Involving Random Variables and Their Expectations 434 14.1 Functions of Random Variables. The Transport Formula 441 Bibliography 445 Index 447