Stochastic Processes

Some Background on Probability


Introduction


Probability


Conditional probability and independence


Discrete random variables


Continuous random variables


Mean and variance


Some standard discrete probability distributions


Some standard continuous probability distributions


Generating functions


Conditional expectation





Some Gambling Problems


Gambler's ruin


Probability of ruin


Some numerical simulations


Duration of the game


Some variations of gambler's ruin





Random Walks


Introduction


Unrestricted random walks


The probability distribution after n steps


First returns of the symmetric random walk





Markov Chains


States and transitions


Transition probabilities


General two-state Markov chains


Powers of the transition matrix for the m-state chain


Gambler's ruin as a Markov chain


Classification of states


Classification of chains





Poisson Processes


Introduction


The Poisson process


Partition theorem approach


Iterative method


The generating function


Variance in terms of the probability generating function


Arrival times


Summary of the Poisson process





Birth and Death Processes


Introduction


The birth process


Birth process: Generating function equation


The death process


The combined birth and death process


General population processes





Queues


Introduction


The single-server queue


The stationary process


Queues with multiple servers


Queues with fixed service times


Classification of queues


A general approach to the M(?)/G/1 queue





Reliability and Renewal


Introduction


The reliability function


Exponential distribution and reliability


Mean time to failure


Reliability of series and parallel systems


Renewal processes


Expected number of renewals





Branching and Other Random Processes


Introduction


Generational growth


Mean and variance


Probability of extinction


Branching processes and martingales


Stopping rules


The simple epidemic


An iterative solution scheme for the simple epidemic





Computer Simulations and Projects





Answers and Comments on End-of-Chapter Problems





Appendix





References and Further Reading





Index


Problems appear at the end of each chapter.