A First Course in Stochastic Processes

PrefaceChapter 1 Elements of Stochastic Processes 1. Review of Basic Terminology and Properties of Random Variables and Distribution Functions 2. Two Simple Examples of Stochastic Processes 3. Classification of General Stochastic Processes Problems ReferencesChapter 2 Markov Chains 1. Definitions 2. Examples of Markov Chains 3. Transition Probability Matrices of a Markov Chain 4. Classification of States of a Markov Chain 5. Recurrence 6. Examples of Recurrent Markov Chains 7. More on Recurrence Problems ReferencesChapter 3 The Basic Limit Theorem of Markov Chains and Applications 1. Discrete Renewal Equation 2. Proof of Theorem 1.1 3. Absorption Probabilities 4. Criteria for Recurrence 5. A Queueing Example 6. Another Queueing Model 7. Random Walk Problems ReferencesChapter 4 Algebraic Methods in Markov Chains 1. Preliminaries 2. Relations of Eigenvalues and Recurrence Classes 3. Periodic Classes 4. Special Computational Methods in Markov Chains 5. Examples 6. Applications to Coin Tossing Problems ReferencesChapter 5 Ratio Theorems of Transition Probabilities and Applications 1. Taboo Probabilities 2. Ratio Theorems 3. Existence of Generalized Stationary Distributions 4. Interpretation of Generalized Stationary Distributions 5. Regular, Superregular, and Subregular Sequences for Markov Chains Problems ReferencesChapter 6 Sums of Independent Random Variables as a Markov Chain 1. Recurrence Properties of Sums of Independent Random Variables 2. Local Limit Theorems 3. Right Regular Sequences for the Markov Chain {Sn} Problems ReferencesChapter 7 Classical Examples of Continuous Time Markov Chains 1. General Pure Birth Processes and Poisson Processes 2. More about Poisson Processes 3. A Counter Model 4. Birth and Death Processes 5. Differential Equations of Birth and Death Processes 6. Examples of Birth and Death Processes 7. Birth and Death Processes with Absorbing States 8. Finite State Continuous Time Markov Chains Problems ReferencesChapter 8 Continuous Time Markov Chains 1. Differentiability Properties of Transition Probabilities 2. Conservative Processes and the Forward and Backward Differential Equations 3. Construction of a Continuous Time Markov Chain from Its Infinitesimal Parameters 4. Strong Markov Property Problems ReferencesChapter 9 Order Statistics, Poisson Processes, and Applications 1. Order Statistics and Their Relation to Poisson Processes 2. The Ballot Problem 3. Empirical Distribution Functions and Order Statistics 4. Some Limit Distributions for Empirical Distribution Functions Problems ReferencesChapter 10 Brownian Motion 1. Background Material 2. Joint Probabilities for Brownian Motion 3. Continuity of Paths and the Maximum Variables Problems ReferencesChapter 11 Branching Processes 1. Discrete Time Branching Processes 2. Generating Function Relations for Branching Processes 3. Extinction Probabilities 4. Examples 5. Two-Type Branching Processes 6. Multi-Type Branching Processes 7. Continuous Time Branching Processes 8. Extinction Probabilities for Continuous Time Branching Processes 9. Limit Theorems for Continuous Time Branching Processes 10. Two-Type Continuous Time Branching Process 11. Branching Processes with General Variable Lifetime Problems ReferencesChapter 12 Compounding Stochastic Processes 1. Multidimensional Homogeneous Poisson Processes 2. An Application of Multidimensional Poisson Processes to Astronomy 3. Immigration and Population Growth 4. Stochastic Models of Mutation and Growth 5. One-Dimensional Geometric Population Growth 6. Stochastic Population Growth Model in Space and Time 7.