1
Background/Review of Statistical Concepts

Chapter Overview

This chapter gives an overview of some key statistical concepts as they relate to statistical process control (SPC). This will aid in familiarizing the reader with concepts and terminology that will be helpful in reading the following chapters.

1.1 Basic Probability

The term probability indicates how likely an event is or what the chance is that the event will happen. Most events can't be predicted with total certainty; the best we can do is say how likely they are to happen, and quantify that likelihood or chance using the concept of probability. A probability is a real number between (and including) zero and one. When an event is certain to happen, its probability equals one, whereas when it is impossible for the event to happen, its probability equals zero. Otherwise, the event is likely to happen or occur with a certain probability, expressed as a fraction between zero and one. For example, when a coin is tossed, there are two possible outcomes, namely, that a head (H) or a tail (T) can be observed. Note that an outcome is the result of a single trial of an experiment and the sample space (S) constitutes all possible outcomes of an experiment (the sample space is exhaustive). In the coin tossing example, the sample space is given by S = {H,T}. If the coin is unbiased (or fair), the probability (P) of observing a head is the same as the probability of observing a tail, each of which equals . The probability of the set of all possible experimental outcomes in the sample space must equal one. In this example, this is evident since P(H) + P(T) = 0.5 + 0.5 = 1. When all experimental outcomes in the sample space are equally likely, this is referred to as the classical method of assigning probabilities, which is illustrated in the coin example. Another example of the classical method of assigning probabilities is when a dice is thrown. In this case, the sample space is given by S = {1,2,3,4,5,6} and if the dice is unbiased (or fair) the probability of observing a one on the dice is the same as observing any other value on the dice that equals . Mathematically, we can write

where Ei defines the ith experimental outcome, i.e.

E1 = 1

Observed value on the dice is a one

E2 = 2

Observed value on the dice is a two

E3 = 3

Observed value on the dice is a three

E4 = 4

Observed value on the dice is a four

E5 = 5

Observed value on the dice is a five

E6 = 6

Observed value on the dice is a six

Again, note that the probability of the set of possible experimental outcomes equals one since

In the two examples given above, the experimental outcomes are equally likely. Let's consider an experiment where the experimental outcomes are not equally likely. Suppose that a glass jar contains four red, eight green, three blue, and five yellow marbles. If a single marble is chosen at random from the jar, what is the probability of choosing a specific color, say, a red marble? In general, the probability of an event occurring is calculated by dividing the number of ways an event can occur by the total number of possible experimental outcomes.

• P(Red) = (Number of red marbles)/(Total number of marbles) =
• P(Green) = (Number of green marbles)/(Total number of marbles) =
• P(Blue) = (Number of blue marbles)/(Total number of marbles) =
• P(Yellow) = (Number of yellow marbles)/(Total number of marbles) =

Again, note that the probability of the set of possible experimental outcomes equals one since

When all the experimental outcomes are not equally likely, this is referred to as the relative frequency method of assigning probabilities, which is illustrated in the marble example.

Next, we consider random variables and their distributions that play the most important roles in statistics and probability.

1.2 Random Variables and Their Distributions

A random variable, denoted as , can take on a value, or, an interval of values, with an associated probability. The random variable can be univariate (one) or bivariate (two) or even multivariate (more than two). There are two major types of random variables, namely, discrete and continuous. Although there are situations where there can be a mixed random variable, which is partly discrete and partly continuous, we focus on the discrete and continuous variables here. To illustrate a discrete random variable, let's consider the coin example where either a head or a tail can be observed in a trial (a coin toss). Suppose that a coin is tossed five times and the random variable denotes the number of heads that are observed. Then can only take on integer values S = {0,1,2,3,4,5} and, accordingly, is a discrete random variable. Another example of a discrete random variable would be an that denotes the number of members in a household. Alternatively, a continuous random variable can take on values within some range. The probability of a continuous variable taking on any specific value is zero. If denotes the height of a tree, then it is possible for a tree to have a height of 2.176 m or even 2.1765482895 m; the number of decimal places depends on the accuracy of the measuring instrument. Thus can take on values other than only integer values, within some range of values and, accordingly, is a continuous random variable. Another example of a continuous random variable would be if denotes the lifetime of a light bulb.

A random variable has an associated probability mass function (pmf) if discrete or a probability density function (pdf) if continuous. First, we define the cumulative distribution function (cdf) before defining the pmf and pdf for discrete and continuous random variables, respectively.

Every random variable has a cumulative distribution function (cdf) that defines its distribution. The cdf is a function that gives the probability that a random variable is less than or equal to some real value , that is, . In the case of a discrete random variable, the cdf is calculated by adding the probabilities up to and including , whereas for a continuous random variable, the cdf is calculated by finding the area (integrating) under its pdf up to . The cdf is a monotone non-decreasing right-continuous function, which is a step function for a discrete random variable (see Figure 1.1) and is a continuous function for a continuous random variable (see Figure 1.2). For Figure 1.1 , it should be noted that with are the discrete values that the random variable can take on. For more details on the properties of a cdf see any mathematical statistics book.

Figure 1.1 The cdf for a discrete random variable.

Figure 1.2 The cdf for a continuous random variable.

The pmf of a discrete random variable is a function that gives the probability that the random variable takes on the value of , that is, . More formally, let , satisfying the following two conditions

The pdf of a continuous random variable is the first derivative of the cdf . That is, . More formally, the pdf must satisfy the following two conditions

The cdf (or equivalently the pmf and the pdf) describes the distribution of a random variable over its values or its range or domain of values, that is, how the total probability (which equals one) is distributed or spread out over the values or the range of values of the random variable(s). Probabilities may be either marginal, joint, or conditional. A marginal probability is the probability of the occurrence of a single event. It may be thought of as an unconditional probability since it is not conditioned (or dependent) on another event. An example of a marginal probability is the probability that a red card is drawn from a deck of cards, which is given by (Red) = , since 26 out of 52 cards, that is, half the cards in a deck of cards, are red. A joint probability is the probability of the joint occurrence (or the intersection) of at least two events. The probability of the intersection of two events, and , may be written as , for example, the probability of drawing a red ace from a deck of cards is given by (Red Ace) = , since there are two red aces in a deck of 52 cards, namely, the ace of hearts and the ace of diamonds. A conditional probability is the probability of event occurring, given that event occurs, and is denoted by , for example, the probability of drawing an ace, given that the card is red, is given by (Ace?|?Red) = , since there are two aces in the total of 26 red cards, namely, the ace of hearts and the ace of diamonds. The definition of a conditional probability is given by

This formula shows the relationship between the marginal, the joint, and the conditional probability. Returning to the example of the deck of cards, for example, (Ace?|?Red) = , which is the same answer as found previously. Typically, a marginal probability relates to an event associated with a single (scalar) random variable, whereas both joint and conditional probabilities relate to events associated with two or more random variables, that is, in a...