1
Stochastic Processes

James R. Cruise, Ostap O. Hryniv, and Andrew R. Wade

1.1 Introduction

Basic probability theory deals, among other things, with random variables and their properties. A random variable is the mathematical abstraction of the following concept: we make a measurement on some physical system, subject to randomness or uncertainty, and observe the value. We can, for example, construct a mathematical model for the system and try to predict the behavior of our random observable, perhaps through its distribution, or at least itsaverage value (mean). Even in the simplest applications, however, we are confronted by systems that change over time. Now we do not have a single random variable, but a family of random variables. The nature of the physical system that we are modeling determines the structure of dependencies of the variables.

A stochastic (or random) process is the mathematical abstraction of these systems that change randomly over time. Formally, a stochastic process is a family of random variables , where T is some index set representing time. The two main examples are (discrete time) and (continuous time); different applications will favor one or other of these. Interesting classes of processes are obtained by imposing additional structure on the family Xt, as we shall see.

The aim of this chapter is to give a tour of some of the highlights of stochastic process theory and its applications in the physical sciences. In line with the intentions of this volume, our emphasis is on tools. However, the combination of a powerful tool and an unsteady grip is a hazardous one, so we have attempted to maintain mathematical accuracy. For reasons of space, the presentation is necessarily concise. While we cover several important topics, we omit many more. We include references for further reading on the topics that we do cover throughout the text and in Section 1.8. The tools that we exhibit include generating functions and other transforms, and renewal structure, including the Markov property, which can be viewed loosely as a notion of statistical self-similarity.

In the next section, we discuss some of the tools that we will use, with some examples. The basic notions of probability theory that we use are summarized in Section 1.A.

1.2 Generating Functions and Integral Transforms

1.2.1 Generating Functions

Given a sequence of real numbers, the function

is called the generating function of . When is finite for some s ? 0, the series (1.1) converges in the disc and thus its coefficients ak can be recovered via differentiation

or using the Cauchy integral formula

with properly chosen r > 0. This observation is often referred to as the uniqueness property: if two generating functions, say and , are finite and coincide in some open neighborhood of the origin, then for all k = 0. In particular, one can identify the sequence from its generating function.

The generating function is one of many transforms very useful in applications. We will discuss some further examples in Section 1.2.3.

Example 1.1

Imagine one needs to pay the sum of n pence using only one pence and two pence coins. In how many ways can this be done?1

1.   If the order matters, that is, when 1 + 2 and 2 + 1 are two distinct ways of paying 3 pence, the question is about counting the number an of monomer/dimer configurations on the interval of length n. One easily sees that , and in general , because the leftmost monomer can be extended to a full configuration in ways, whereas the leftmost dimer is compatible with exactly configurations on the remainder of the interval. A straightforward computation using the recurrence relation above now gives
2. so that . The coefficient an can now be recovered using (1.2), (1.3), or partial fractions and then power series expansion.
3.   If the order does not matter, that is, when 1 + 2 and 2 + 1 are regarded as identical ways of paying 3 pence, the question above boils down to calculating the number bn of nonnegative integer solutions to the equation . One easily sees that , and a straightforward induction shows that bn is the coefficient of sk in the product
4. In other words, .

In this chapter, we will make use of generating functions for various sequences with probabilistic meaning. In particular, given a -valued2 random variable X, we can consider the corresponding probability generating function, which is the generating function of the sequence , where , describes the probability mass function of X. Thus, the probability generating function of X is given by (writing