
Applied Linear Regression
Description
Alles über E-Books | Antworten auf Fragen rund um E-Books, Kopierschutz und Dateiformate finden Sie in unserem Info- & Hilfebereich.
More details
Other editions
Additional editions

Person
Content
1 Scatterplots 1
1.1 Scatterplots 2
1.2 Mean Functions 9
1.3 Variance Functions 12
1.4 Summary Graph 12
1.5 Tools for Looking at Scatterplots 13
1.6 Scatterplot Matrices 15
1.7 Problems 17
2 Simple Linear Regression 21
2.1 Ordinary Least Squares Estimation 22
2.2 Least Squares Criterion 24
2.3 Estimating the Variance ¿¿¿¿2 26
2.4 Properties of Least Squares Estimates 27
2.5 Estimated Variances 28
2.6 Confidence Intervals and ¿¿¿¿-Tests 29
2.7 The Coefficient of Determination, ¿¿¿¿2 33
2.8 The Residuals 35
2.9 Problems 37
3 Multiple Regression 49
3.1 Adding a Regressor to a Simple Linear Regression Model 49
3.2 The Multiple Linear Regression Model 53
3.3 Predictors and Regressors 53
3.4 Ordinary Least Squares 57
3.5 Predictions, Fitted Values and Linear Combinations 65
3.6 Problems 66
4 Interpretation of Main Effects 71
4.1 Understanding Parameter Estimates 71
4.2 Dropping Regressors 81
4.3 Experimentation Versus Observation 84
4.4 Sampling from a Normal Population 86
4.5 More on ¿¿¿¿2 88
4.6 Problems 90
5 Complex Regressors 95
5.1 Factors 95
5.2 Many Factors 105
5.3 Polynomial Regression 106
5.4 Splines 109
5.5 Principal Components 112
5.6 Missing Data 115
5.7 Problems 118
6 Testing and Analysis of Variance 129
6.1 ¿¿¿¿-tests 130
6.2 The Analysis of Variance 134
6.3 Comparisons of Means 138
6.4 Power and Non-null Distributions 138
6.5 Wald Tests 140
6.6 Interpreting Tests 142
6.7 Problems 145
7 Variances 151
7.1 Weighted Least Squares 151
7.2 Misspecified Variances 157
7.3 General Correlation Structures 162
7.4 Mixed Models 163
7.5 Variance Stabilizing Transformations 165
7.6 The Delta Method 166
7.7 The Bootstrap 168
7.8 Problems 173
8 Transformations 179
8.1 Transformation Basics 179
8.2 A General Approach to Transformations 185
8.3 Transforming the Response 190
8.4 Transformations of Nonpositive Variables 192
8.5 Additive Models 192
8.6 Problems 193
9 Regression Diagnostics 199
9.1 The Residuals 199
9.2 Testing for Curvature 206
9.3 Nonconstant Variance 208
9.4 Outliers 208
9.5 Influence of Cases 212
9.6 Normality Assumption 218
9.7 Problems 220
10 Variable Selection 227
10.1 Variable Selection and Parameter Assessment 228
10.2 Variable Selection for Discovery 230
10.3 Model Selection for Prediction 238
10.4 Problems 241
11 Nonlinear Regression 245
11.1 Estimation for Nonlinear Mean Functions 246
11.2 Inference Assuming Large Samples 249
11.3 Starting Values 249
11.4 Bootstrap Inference 255
11.5 Further Reading 257
11.6 Problems 258
12 Binomial and Poisson Regression 263
12.1 Distributions for Counted Data 263
12.2 Regression Models For Counts 265
12.3 Poisson Regression 271
12.4 Transferring What You Know about Linear Models 276
12.5 Generalized Linear Models 278
12.6 Problems 278
A Appendix 283
A.1 Website 283
A.2 Means, Variances, Covariances and Correlations 283
A.3 Least Squares for Simple Regression 286
A.4 Means and Variances of Least Squares Estimates 286
A.5 Estimating E(¿¿¿¿ |¿¿¿¿) using a Smoother 288
A.6 A Brief Introduction to Matrices and Vectors 290
A.7 Random Vectors 295
A.8 Least Squares Using Matrices 295
A.9 The QR factorization 299
A.10 Spectral Decomposition 300
A.11 Maximum Likelihood Estimates 300
A.12 The Box-Cox Method for Transformations 302
A.13 Case Deletion in Linear Regression 305
Bibliography 321
Index 322
CHAPTER 1
Scatterplots and Regression
Regression is the study of dependence. It is used to answer interesting questions about how one or more predictors influence a response. Here are a few typical questions that may be answered using regression:
- Are daughters taller than their mothers?
- Does changing class size affect success of students?
- Can we predict the time of the next eruption of Old Faithful Geyser from the length of the most recent eruption?
- Do changes in diet result in changes in cholesterol level, and if so, do the results depend on other characteristics such as age, sex, and amount of exercise?
- Do countries with higher per person income have lower birth rates than countries with lower income?
- Are highway design characteristics associated with highway accident rates? Can accident rates be lowered by changing design characteristics?
- Is water usage increasing over time?
- Do conservation easements on agricultural property lower land value?
In most of this book, we study the important instance of regression methodology called linear regression. This method is the most commonly used in regression, and virtually all other regression methods build upon an understanding of how linear regression works.
As with most statistical analyses, the goal of regression is to summarize observed data as simply, usefully, and elegantly as possible. A theory may be available in some problems that specifies how the response varies as the values of the predictors change. If theory is lacking, we may need to use the data to help us decide on how to proceed. In either case, an essential first step in regression analysis is to draw appropriate graphs of the data.
We begin in this chapter with the fundamental graphical tools for studying dependence. In regression problems with one predictor and one response, the scatterplot of the response versus the predictor is the starting point for regression analysis. In problems with many predictors, several simple graphs will be required at the beginning of an analysis. A scatterplot matrix is a convenient way to organize looking at many scatterplots at once. We will look at several examples to introduce the main tools for looking at scatterplots and scatterplot matrices and extracting information from them. We will also introduce notation that will be used throughout the book.
1.1 Scatterplots
We begin with a regression problem with one predictor, which we will generically call X, and one response variable, which we will call Y.1 Data consist of values (xi, yi), i = 1, … , n, of (X, Y) observed on each of n units or cases. In any particular problem, both X and Y will have other names that will be displayed in this book using typewriter font, such as temperature or concentration, that are more descriptive of the data that are to be analyzed. The goal of regression is to understand how the values of Y change as X is varied over its range of possible values. A first look at how Y changes as X is varied is available from a scatterplot.
Inheritance of Height
One of the first uses of regression was to study inheritance of traits from generation to generation. During the period 1893–1898, Karl Pearson (1857–1936) organized the collection of n = 1375 heights of mothers in the United Kingdom under the age of 65 and one of their adult daughters over the age of 18. Pearson and Lee (1903) published the data, and we shall use these data to examine inheritance. The data are given in the data file Heights.2
Our interest is in inheritance from the mother to the daughter, so we view the mother's height, called mheight, as the predictor variable and the daughter's height, dheight, as the response variable. Do taller mothers tend to have taller daughters? Do shorter mothers tend to have shorter daughters?
A scatterplot of dheight versus mheight helps us answer these questions. The scatterplot is a graph of each of the n points with the response dheight on the vertical axis and predictor mheight on the horizontal axis. This plot is shown in Figure 1.1a. For regression problems with one predictor X and a response Y, we call the scatterplot of Y versus X a summary graph.
Figure 1.1 Scatterplot of mothers' and daughters' heights in the Pearson and Lee data. The original data have been jittered to avoid overplotting in (a). Plot (b) shows the original data, so each point in the plot refers to one or more mother–daughter pairs.
Here are some important characteristics of this scatterplot:
1. The range of heights appears to be about the same for mothers and for daughters. Because of this, we draw the plot so that the lengths of the horizontal and vertical axes are the same, and the scales are the same. If all mothers and daughters pairs had exactly the same height, then all the points would fall exactly on a 45°-line. Some computer programs for drawing a scatterplot are not smart enough to figure out that the lengths of the axes should be the same, so you might need to resize the plot or to draw it several times. 2. The original data that went into this scatterplot were rounded so each of the heights was given to the nearest inch. The original data are plotted in Figure 1.1b. This plot exhibits substantial overplotting with many points at exactly the same location. This is undesirable because one point on the plot can correspond to many cases. The easiest solution is to use jittering, in which a small uniform random number is added to each value. In Figure 1.1a, we used a uniform random number on the range from −0.5 to +0.5, so the jittered values would round to the numbers given in the original source. 3. One important function of the scatterplot is to decide if we might reasonably assume that the response on the vertical axis is independent of the predictor on the horizontal axis. This is clearly not the case here since as we move across Figure 1.1a from left to right, the scatter of points is different for each value of the predictor. What we mean by this is shown in Figure 1.2, in which we show only points corresponding to mother–daughter pairs with mheight rounding to either 58, 64, or 68 inches. We see that within each of these three strips or slices, the number of points is different, and the mean of dheight is increasing from left to right. The vertical variability in dheight seems to be more or less the same for each of the fixed values of mheight. 4. In Figure 1.1a the scatter of points appears to be more or less elliptically shaped, with the major axis of the ellipse tilted upward, and with more points near the center of the ellipse rather than on the edges. We will see in Section 1.4 that summary graphs that look like this one suggest the use of the simple linear regression model that will be discussed in Chapter 2. 5. Scatterplots are also important for finding separated points. Horizontal separation would occur for a value on the horizontal axis mheight that is either unusually small or unusually large relative to the other values of mheight. Vertical separation would occur for a daughter with dheight either relatively large or small compared with the other daughters with about the same value for mheight. These two types of separated points have different names and roles in a regression problem. Extreme values on the left and right of the horizontal axis are points that are likely to be important in fitting regression models and are called leverage points. The separated points on the vertical axis, here unusually tall or short daughters give their mother's height, are potentially outliers, cases that are somehow different from the others in the data. Outliers are more easily discovered in residual plots, as illustrated in the next example. While the data in Figure 1.1a do include a few tall and a few short mothers and a few tall and short daughters, given the height of the mothers, none appears worthy of special treatment, mostly because in a sample size this large, we expect to see some fairly unusual mother–daughter pairs.Figure 1.2 Scatterplot showing only pairs with mother's height that rounds to 58, 64, or 68 inches.
Forbes's Data
In an 1857 article, the Scottish physicist James D. Forbes (1809–1868) discussed a series of experiments that he had done concerning the relationship between atmospheric pressure and the boiling point of water. He knew that altitude could be determined from atmospheric pressure, measured with a barometer, with lower pressures corresponding to higher altitudes. Barometers in the middle of the nineteenth century were fragile instruments, and Forbes wondered if a simpler measurement of the boiling point of water could substitute for a direct reading of barometric pressure. Forbes collected data in the Alps and in Scotland. He measured at each location the atmospheric pressure pres in inches of mercury with a barometer and boiling point...
System requirements
File format: ePUB
Copy protection: Adobe-DRM (Digital Rights Management)
System requirements:
- Computer (Windows; MacOS X; Linux): Install the free reader Adobe Digital Editions prior to download (see eBook Help).
- Tablet/smartphone (Android; iOS): Install the free app Adobe Digital Editions or the app PocketBook before downloading (see eBook Help).
- E-reader: Bookeen, Kobo, Pocketbook, Sony, Tolino and many more (not Kindle).
The file format ePub works well for novels and non-fiction books – i.e., „flowing” text without complex layout. On an e-reader or smartphone, line and page breaks automatically adjust to fit the small displays.
This eBook uses Adobe-DRM, a „hard” copy protection. If the necessary requirements are not met, unfortunately you will not be able to open the eBook. You will therefore need to prepare your reading hardware before downloading.
Please note: We strongly recommend that you authorise using your personal Adobe ID after installation of any reading software.
For more information, see our ebook Help page.