Introduction to Probability Models

Part 1 Introduction to probability theory: sample space and events; probabilities defined on events; conditional probabilities; independent events; Bayes' formula. Part 2 Random variables: random variables; discrete random variables - the Bernoulli random variable; the binomial random variable; the geometric random variable; the Poisson random variable; continuous random variables - the uniform random variable; exponential random variables; gamma random variables; normal random variables; expectation of a random variable - the discrete case; the continuous case; expectation of a function of a random variable; jointly distributed random variables - joint distribution functions; independent random variables; joint probability distribution of functions of random variables; moment generating functions; limit theorems; stochastic processes. Part 3 Conditional probability and conditional expectation: the discrete case; the continuous case; computing expectations by conditioning; computing probabilities by conditioning; some applications - a list model; a random graph; uniform priors, Polya's Um model and Bose-Einstein statistics; in normal sampling X and S2 are independent problems. Part 4 Markov chains: Chapman-Kolmogorov equations; classification of states; limiting probabilities; some applications - the Gambler's Ruin problem; a model for algorithmic efficiency; branching processes; time reversible Markov chains; Markov decision processes. Part 5 The exponential distribution and the Poisson process: the exponential distribution - definition; properties of the exponential distribution; further properties of the exponential distribution; the Poisson process - counting processes; definition of the Poisson process; interarrival and waiting time distributions; further properties of Poisson processes; conditional distribution of the arrival times; estimating software reliability; generalizations of the Poisson process - nonhomogeneous Poisson process; compound Poisson process. Part 6 Continuous-time Markov chains: continuous-time Markov chains; birth and death processes; the Kolmogorov differential equations; limiting probabilities; time reversibility; uniformization; computing the transition probabilities. Part 7 Renewal theory and its applications: distribution of N(t); limit theorems and their applications; renewal reward processes. (Part contents)