Chapter One

Probability Space

1.1 Introduction/Purpose of the Chapter

The most important object when working with probability is the proper definition of the space studied. Typically, one wants to obtain answers about real-life phenomena which do not have a predetermined outcome. For example, when playing a complex game a person may be wondering: What are my chances to win this game? Or, am I paying too much to play this game, and is there perhaps a different game I should rather play? A certain civil engineer wants to know what is the probability that a particular construction material will fail under a lot of stress. To be able to answer these and other questions, we need to make the transition from reality to a space describing what may happen and to create consistent laws on that space. This framework allows the creation of a mathematical model of the random phenomena. This model, should it be created in the proper (consistent) way, will allow the modeler to provide approximate answers to the relevant questions asked. Thus, the first and the most important step in creating consistent models is to define a probability space which is capable of answering the interesting questions that may be asked.

We denote with Ω the set that contains all the possible outcomes of a random experiment. The set Ω is often called sample space or universal sample space. For example, if one rolls a die, Ω = {1, 2, 3, 4, 5, 6}. The space Ω does not necessarily contain numbers but rather some representation of the outcomes of the real phenomena. For example, if one looks at the types of bricks which may be used to build a house, a picture of each possible brick is a possible representation of each element of Ω.

A generic element of Ω will be denoted by ω. Any collection of outcomes (elements in Ω) is called an event. That is, an event is any subset of the sample space Ω. We will use capital letters from the beginning of the alphabet (A, B, C, etc.) to denote events.

In probability, one needs to measure the size of these events. Since an event is just a subset, we need to define those subsets of 3 Ω that can be measured. The concept of sigma algebra allows us to define a collection of subsets of the sample space on which a measure can be defined. In this chapter we introduce the notion of algebra and sigma algebra and we discuss their basic properties.

1.2 Vignette/Historical Notes

The first recorded notions of Probability Theory appear in 1654 in an exchange of letters between the famous mathematicians Blaise Pascal and Pierre de Fermat. The correspondence was prompted by a simple observation by Antoine Gombaud Chevalier de Méré, a French nobleman with an interest in gambling and who was puzzled by an apparent contradiction concerning a popular dice game. The game consisted of throwing a pair of dice 24 times; the problem was to decide whether or not to bet even money on the occurrence of at least one “double six” during the 24 throws. A seemingly well-established gambling rule led de Méré to believe that betting on a double six in 24 throws would be profitable (based on the payoff of the game). However, his own calculations based on many repetitions of the 24 throws indicated just the opposite. If we translate de Méré problem in todays language, he was trying to establish if the event has probability greater than 0.5. Today the confusion is easy to pinpoint to the proper definition of the probability space. For example, the convention at the time was that rolling a three (two and one showing on the dice) would be the same as rolling a two (a double one). Puzzled by this and other similar gambling problems, de Méré wrote to Pascal. The further correspondence between Fermat and Pascal is the first known documentation of the fundamental principles of the theory of probability.

The first formal treatment of probability theory was provided by Pierre-Simon, marquis de Laplace (1749–1827) in his Théorie Analytique des Probabilités published in 1812. (Laplace, 1886, republished). In 1933 the monograph Grundbegriffe der Wahrscheinlichkeitsrechnung by the Russian preeminent mathematician Andrey Nikolaevich Kolmogorov (1903–1987) outlined the axiomatic approach that forms the basis of the modern probability theory as we know it today (Kolmogoroff, 1973, republished).

1.3 Notations and Definitions

The following notations will be used throughout the book for set (event) operations. In the following, ω is any element and A, B are any sets in the sample space Ω.

• We describe a set or a collection of elements using a notation of the form

• ø is a notation for the set that does not contain any element. This set is called the empty set.
• ω A denotes that the element ω is in the set A; we say “ω belongs to A.” Obviously, ω ∉ A means that the element is not in the set.
• A ⊆ B denotes that A is a subset of B; that is, every element in A is also in B. The set A may actually be equal to B. In contrast, the notation A ⊂ B means that A is a proper subset of B; that is, A is strictly included in B. Mathematically, A ⊆ B is equivalent to the following statement: Any x A implies x B.
• Union of sets:

• Intersection of sets:

• Complement of a set:

• Difference of two sets:

• Symmetric difference:

• Two sets A, B such that A∩ B = ø are called disjoint or mutually exclusive sets.
• A collection of sets A1, A2, …, An such that A1 ∪ A2 ∪ … ∪ An = Ω and Ai∩ Aj = ø for any i ≠ j is called a partition of the space Ω.

Every set operation may be expressed in terms of basic operations. For example,

There is a distributive law for intersection over union. If A, B, C are included in Ω, then

and

Furthermore, Ωc =ø and øc = Ω, and for all A ⊂ Ω we have

The De Morgan laws are also very important:

(1.1)

All of these rules may be extended to any finite number of sets in an obvious way. More details and further references about set operations may be found in Billingsley (1995) or Chung (2000).

1.4 Theory and Applications

1.4.1 Algebras

We introduce the notion of σ-algebra (or σ-field) to introduce a collection of sets which we may measure. In other words, we introduce a proper domain of definition for the (soon to be introduced) probability function. First let us denote

that is, is the collection of all possible subsets of Ω, a set containing all possible sets in Ω. This collection is called the parts of Ω.

An algebra on Ω is a collection of such sets in (including Ω) which is closed under complementarity and finite union.

Let us list some immediate property of an algebra.

Proof: Note that these are properties of the collection of sets. The collection must contain these. Specifically, ø = Ωc and since Ω the second point in Definition 1.1 says that ø . Since

by the DeMorgan laws...