Probability and Measure

FOREWORD xi PREFACE xiii Patrick Billingsley 1925-2011 xv Chapter1 PROBABILITY 1 1. BOREL'S NORMAL NUMBER THEOREM, 1 The Unit Interval The Weak Law of Large Numbers The Strong Law of Large Numbers Strong Law Versus Weak Length The Measure Theory of Diophantine Approximation* 2. PROBABILITY MEASURES, 18 Spaces Assigning Probabilities Classes of Sets Probability Measures Lebesgue Measure on the Unit Interval Sequence Space* Constructing s-Fields* 3. EXISTENCE AND EXTENSION, 39 Construction of the Extension Uniqueness and the p-? Theorem Monotone Classes Lebesgue Measure on the Unit Interval Completeness Nonmeasurable Sets Two Impossibility Theorems* 4. DENUMERABLE PROBABILITIES, 53 General Formulas Limit Sets Independent Events Subfields The Borel-Cantelli Lemmas The Zero-One Law 5. SIMPLE RANDOM VARIABLES, 72 Definition Convergence of Random Variables Independence Existence of Independent Sequences Expected Value Inequalities 6. THE LAW OF LARGE NUMBERS, 90 The Strong Law The Weak Law Bernstein's Theorem A Refinement of the Second Borel-Cantelli Lemma 7. GAMBLING SYSTEMS, 98 Gambler's Ruin Selection Systems Gambling Policies Bold Play* Timid Play* 8. MARKOV CHAINS, 117 Definitions Higher-Order Transitions An Existence Theorem Transience and Persistence Another Criterion for Persistence Stationary Distributions Exponential Convergence* Optimal Stopping* 9. LARGE DEVIATIONS AND THE LAW OF THE ITERATED LOGARITHM, 154 Moment Generating Functions Large Deviations Chernoff's Theorem* The Law of the Iterated Logarithm Chapter2 MEASURE 167 10. GENERAL MEASURES, 167 Classes of Sets Conventions Involving 8 Measures Uniqueness 11. OUTER MEASURE, 174 Outer Measure Extension An Approximation Theorem 12. MEASURES IN EUCLIDEAN SPACE, 181 Lebesgue Measure Regularity Specifying Measures on the Line Specifying Measures in Rk Strange Euclidean Sets* 13. MEASURABLE FUNCTIONS AND MAPPINGS, 192 Measurable Mappings Mappings into Rk Limits and Measurability Transformations of Measures 14. DISTRIBUTION FUNCTIONS, 198 Distribution Functions Exponential Distributions Weak Convergence Convergence of Types* Extremal Distributions* Chapter3 INTEGRATION 211 15. THE INTEGRAL, 211 Definition Nonnegative Functions Uniqueness 16. PROPERTIES OF THE INTEGRAL, 218 Equalities and Inequalities Integration to the Limit Integration over Sets Densities Change of Variable Uniform Integrability Complex Functions 17. THE INTEGRAL WITH RESPECT TO LEBESGUE MEASURE, 234 The Lebesgue Integral on the Line The Riemann Integral The Fundamental Theorem of Calculus Change of Variable The Lebesgue Integral in Rk Stieltjes Integrals 18. PRODUCT MEASURE AND FUBINI'S THEOREM, 245 Product Spaces Product Measure Fubini's Theorem Integration by Parts Products of Higher Order 19. THE Lp SPACES*, 256 Definitions Completeness and Separability Conjugate Spaces Weak Compactness Some Decision Theory The Space L2 An Estimation Problem Chapter4 RANDOM VARIABLES AND EXPECTED VALUES 271 20. RANDOM VARIABLES AND DISTRIBUTIONS, 271 Random Variables and Vectors Subfields Distributions Multidimensional Distributions Independence Sequences of Random Variables Convolution Convergence in Probability The Glivenko-Cantelli Theorem* 21. EXPECTED VALUES, 291 Expected Value as Integral Expected Values and Limits Expected Values and Distributions Moments Inequalities Joint Integrals Independence and Expected Value Moment Generating Functions 22. SUMS OF INDEPENDENT RANDOM VARIABLES, 300 The Strong Law of Large Numbers The Weak Law and Moment Generating Functions Kolmogorov's Zero-One Law Maximal Inequalities Convergence of Random Series Random Taylor Series* 23. THE POISSON PROCESS, 316 Characterization of the Exponential Distribution The Poisson Process The Poisson Approximation Other Characterizations of the Poisson Process Stochastic Processes 24. THE ERGODIC THEOREM*, 330 Measure-Preserving Transformations Ergodicity Ergodicity of Rotations Proof of the Ergodic Theorem The Continued-Fraction Transformation Diophantine Approximation Chapter5 CONVERGENCE OF DISTRIBUTIONS 349 25. WEAK CONVERGENCE, 349 Definitions Uniform Distribution Modulo 1* Convergence in Distribution Convergence in Probability Fundamental Theorems Helly's Theorem Integration to the Limit 26. CHARACTERISTIC FUNCTIONS, 365 Definition Moments and Derivatives Independence Inversion and the Uniqueness Theorem The Continuity Theorem Fourier Series* 27. THE CENTRAL LIMIT THEOREM, 380 Identically Distributed Summands The Lindeberg and Lyapounov Theorems Dependent Variables* 28. INFINITELY DIVISIBLE DISTRIBUTIONS*, 394 Vague Convergence The Possible Limits Characterizing the Limit 29. LIMIT THEOREMS IN Rk, 402 The Basic Theorems Characteristic Functions Normal Distributions in Rk The Central Limit Theorem 30. THE METHOD OF MOMENTS*, 412 The Moment Problem Moment Generating Functions Central Limit Theorem by Moments Application to Sampling Theory Application to Number Theory Chapter6 DERIVATIVES AND CONDITIONAL PROBABILITY 425 31. DERIVATIVES ON THE LINE*, 425 The Fundamental Theorem of Calculus Derivatives of Integrals Singular Functions Integrals of Derivatives Functions of Bounded Variation 32. THE RADON-NIKODYM THEOREM, 446 Additive Set Functions The Hahn Decomposition Absolute Continuity and Singularity The Main Theorem 33. CONDITIONAL PROBABILITY, 454 The Discrete Case The General Case Properties of Conditional Probability Difficulties and Curiosities Conditional Probability Distributions 34. CONDITIONAL EXPECTATION, 472 Definition Properties of Conditional Expectation Conditional Distributions and Expectations Sufficient Subfields* Minimum-Variance Estimation* 35. MARTINGALES, 487 Definition Submartingales Gambling Functions of Martingales Stopping Times Inequalities Convergence Theorems Applications: Derivatives Likelihood Ratios Reversed Martingales Applications: de Finetti's Theorem Bayes Estimation A Central Limit Theorem* Chapter7 STOCHASTIC PROCESSES 513 36. KOLMOGOROV'S EXISTENCE THEOREM, 513 Stochastic Processes Finite-Dimensional Distributions Product Spaces Kolmogorov's Existence Theorem The Inadequacy of RT A Return to Ergodic Theory The Hewitt-Savage Theorem* 37. BROWNIAN MOTION, 530 Definition Continuity of Paths Measurable Processes Irregularity of Brownian Motion Paths The Strong Markov Property The Reflection Principle Skorohod Embedding Invariance* 38. NONDENUMERABLE PROBABILITIES, 558 Introduction Definitions Existence Theorems Consequences of Separability* APPENDIX 571 NOTES ON THE PROBLEMS 587 BIBLIOGRAPHY 617 INDEX 619