Applied Regression Modeling
Description
The newly and thoroughly revised 3rd Edition of Applied Regression Modeling delivers a concise but comprehensive treatment of the application of statistical regression analysis for those with little or no background in calculus. Accomplished instructor and author Dr. Iain Pardoe has reworked many of the more challenging topics, included learning outcomes and additional end-of-chapter exercises, and added coverage of several brand-new topics including multiple linear regression using matrices.
The methods described in the text are clearly illustrated with multi-format datasets available on the book's supplementary website. In addition to a fulsome explanation of foundational regression techniques, the book introduces modeling extensions that illustrate advanced regression strategies, including model building, logistic regression, Poisson regression, discrete choice models, multilevel models, Bayesian modeling, and time series forecasting. Illustrations, graphs, and computer software output appear throughout the book to assist readers in understanding and retaining the more complex content. Applied Regression Modeling covers a wide variety of topics, like:
* Simple linear regression models, including the least squares criterion, how to evaluate model fit, and estimation/prediction
* Multiple linear regression, including testing regression parameters, checking model assumptions graphically, and testing model assumptions numerically
* Regression model building, including predictor and response variable transformations, qualitative predictors, and regression pitfalls
* Three fully described case studies, including one each on home prices, vehicle fuel efficiency, and pharmaceutical patches
Perfect for students of any undergraduate statistics course in which regression analysis is a main focus, Applied Regression Modeling also belongs on the bookshelves of non-statistics graduate students, including MBAs, and for students of vocational, professional, and applied courses like data science and machine learning.
More details
Other editions
Additional editions
Previous edition
Person
Iain Pardoe, PhD, received his PhD in Statistics from the University of Minnesota. He is an Online Instructor of the "Regression Methods" graduate course at Pennsylvania State University. He also teaches "Biostatistics," "Mathematics for Computing Science," and "Mathematics for Teachers" at Thompson Rivers University and was previously an Associate Professor at the University of Oregon.
Content
Preface xi
Acknowledgments xv
Introduction xvii
I.1 Statistics in Practice xvii
I.2 Learning Statistics xix
About the Companion Website xxi
1 Foundations 1
1.1 Identifying and Summarizing Data 2
1.2 Population Distributions 5
1.3 Selecting Individuals at Random-Probability 9
1.4 Random Sampling 11
1.4.1 Central limit theorem-normal version 12
1.4.2 Central limit theorem-t-version 14
1.5 Interval Estimation 16
1.6 Hypothesis Testing 20
1.6.1 The rejection region method 20
1.6.2 The p-value method 23
1.6.3 Hypothesis test errors 27
1.7 Random Errors and Prediction 28
1.8 Chapter Summary 31
Problems 31
2 Simple Linear Regression 39
2.1 Probability Model for X and Y 40
2.2 Least Squares Criterion 45
2.3 Model Evaluation 50
2.3.1 Regression standard error 51
2.3.2 Coefficient of determination-R2 53
2.3.3 Slope parameter 57
2.4 Model Assumptions 65
2.4.1 Checking the model assumptions 66
2.4.2 Testing the model assumptions 72
2.5 Model Interpretation 72
2.6 Estimation and Prediction 74
2.6.1 Confidence interval for the population mean, E(Y) 74
2.6.2 Prediction interval for an individual Y -value 75
2.7 Chapter Summary 79
2.7.1 Review example 80
Problems 83
3 Multiple Linear Regression 95
3.1 Probability Model for (X1, X2, . . .) and Y 96
3.2 Least Squares Criterion 100
3.3 Model Evaluation 106
3.3.1 Regression standard error 106
3.3.2 Coefficient of determination-R2 108
3.3.3 Regression parameters-global usefulness test 115
3.3.4 Regression parameters-nested model test 120
3.3.5 Regression parameters-individual tests 127
3.4 Model Assumptions 137
3.4.1 Checking the model assumptions 137
3.4.2 Testing the model assumptions 143
3.5 Model Interpretation 145
3.6 Estimation and Prediction 146
3.6.1 Confidence interval for the population mean, E(Y ) 147
3.6.2 Prediction interval for an individual Y -value 148
3.7 Chapter Summary 151
Problems 152
4 Regression Model Building I 159
4.1 Transformations 161
4.1.1 Natural logarithm transformation for predictors 161
4.1.2 Polynomial transformation for predictors 167
4.1.3 Reciprocal transformation for predictors 171
4.1.4 Natural logarithm transformation for the response 175
4.1.5 Transformations for the response and predictors 179
4.2 Interactions 184
4.3 Qualitative Predictors 191
4.3.1 Qualitative predictors with two levels 192
4.3.2 Qualitative predictors with three or more levels 201
4.4 Chapter Summary 210
Problems 211
5 Regression Model Building II 221
5.1 Influential Points 223
5.1.1 Outliers 223
5.1.2 Leverage 228
5.1.3 Cook's distance 230
5.2 Regression Pitfalls 234
5.2.1 Nonconstant variance 234
5.2.2 Autocorrelation 237
5.2.3 Multicollinearity 242
5.2.4 Excluding important predictor variables 246
5.2.5 Overfitting 249
5.2.6 Extrapolation 250
5.2.7 Missing data 252
5.2.8 Power and sample size 255
5.3 Model Building Guidelines 256
5.4 Model Selection 259
5.5 Model Interpretation Using Graphics 263
5.6 Chapter Summary 270
Problems 272
Notation and Formulas 287
Univariate Data 287
Simple Linear Regression 288
Multiple Linear Regression 289
Bibliography 293
Glossary 299
Index 305
6 Case studies 533
6.1 Home prices 533
6.1.1 Data description 533
6.1.2 Exploratory data analysis 536
6.1.3 Regression model building 539
6.1.4 Results and conclusions 542
6.1.5 Further questions 551
6.2 Vehicle fuel efficiency 552
6.2.1 Data description 552
6.2.2 Exploratory data analysis 554
6.2.3 Regression model building 556
6.2.4 Results and conclusions 557
6.2.5 Further questions 567
6.3 Pharmaceutical patches 568
6.3.1 Data description 568
6.3.2 Exploratory data analysis 569
6.3.3 Regression model building 570
6.3.4 Model diagnostics 573
6.3.5 Results and conclusions 574
6.3.6 Further questions 578
7 Extensions 579
7.1 Generalized linear models 581
7.1.1 Logistic regression 582
7.1.2 Poisson regression 594
7.2 Discrete choice models 602
7.3 Multilevel models 609
7.4 Bayesian modeling 614
7.4.1 Frequentist inference 614
7.4.2 Bayesian inference 616
Problems 620
A Computer software help 623
Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 626
B Critical values for t-distributions 631
C Notation and formulas 635
C.1 Univariate data 635
C.2 Simple linear regression 637
C.3 Multiple linear regression 639
D Mathematics refresher 643
D.1 The natural logarithm and exponential functions 643
D.2 Rounding and accuracy 644
E Multiple Linear Regression Using Matrices 647
E.1 Vectors and matrices 647
E.2 Matrix multiplication 649
E.3 Matrix addition 652
E.4 Transpose of a matrix 654
E.5 Inverse of a matrix 656
E.6 Estimated multiple linear regression model equation 657
E.7 Least squares regression parameter estimates 659
E.8 Predicted or fitted values 661
E.9 Residuals and the regression standard error 663
E.10 Coefficient of determination 664
E.11 Regression parameter standard errors and t-statistics 665
E.12 Estimation and prediction 666
E.13 Leverages, standardized and studentized residuals, and Cook's distances 668
F Answers for selected problems 673
INTRODUCTION
I.1 STATISTICS IN PRACTICE
Statistics is used in many fields of application since it provides an effective way to analyze quantitative information. Some examples include:
-
A pharmaceutical company is developing a new drug for treating a particular disease more effectively. How might statistics help you decide whether the drug will be safe and effective if brought to market?
Clinical trials involve large-scale statistical studies of people-usually both patients with the disease and healthy volunteers-who are assessed for their response to the drug. To determine that the drug is both safe and effective requires careful statistical analysis of the trial results, which can involve controlling for the personal characteristics of the people (e.g., age, gender, health history) and possible placebo effects, comparisons with alternative treatments, and so on.
-
A manufacturing firm is not getting paid by its customers in a timely manner-this costs the firm money on lost interest. You've collected recent data for the customer accounts on amount owed, number of days since the customer was billed, and size of the customer (small, medium, large). How might statistics help you improve the on-time payment rate?
You can use statistics to find out whether there is an association between the amount owed and the number of days and/or size. For example, there may be a positive association between amount owed and number of days for small and medium-sized customers but not for large-sized customers-thus it may be more profitable to focus collection efforts on small and medium-sized customers billed some time ago, rather than on large-sized customers or customers billed more recently.
-
A firm makes scientific instruments and has been invited to make a sealed bid on a large government contract. You have cost estimates for preparing the bid and fulfilling the contract, as well as historical information on similar previous contracts on which the firm has bid (some successful, others not). How might statistics help you decide how to price the bid?
You can use statistics to model the association between the success/failure of past bids and variables such as bid cost, contract cost, bid price, and so on. If your model proves useful for predicting bid success, you could use it to set a maximum price at which the bid is likely to be successful.
-
As an auditor, you'd like to determine the number of price errors in all of a company's invoices-this will help you detect whether there might be systematic fraud at the company. It is too time-consuming and costly to examine all of the company's invoices, so how might statistics help you determine an upper bound for the proportion of invoices with errors?
Statistics allows you to infer about a population from a relatively small random sample of that population. In this case, you could take a sample of 100 invoices, say, to find a proportion, p, such that you could be 95% confident that the population error rate is less than that quantity p.
-
A firm manufactures automobile parts and the factory manager wants to get a better understanding of overhead costs. You believe two variables in particular might contribute to cost variation: machine hours used per month and separate production runs per month. How might statistics help you to quantify this information?
You can use statistics to build a multiple linear regression model that estimates an equation relating the variables to one another. Among other things you can use the model to determine how much cost variation can be attributed to the two cost drivers, their individual effects on cost, and predicted costs for particular values of the cost drivers.
-
You work for a computer chip manufacturing firm and are responsible for forecasting future sales. How might statistics be used to improve the accuracy of your forecasts?
Statistics can be used to fit a number of different forecasting models to a time series of sales figures. Some models might just use past sales values and extrapolate into the future, while others might control for external variables such as economic indices. You can use statistics to assess the fit of the various models, and then use the best-fitting model, or perhaps an average of the few best-fitting models, to base your forecasts on.
-
As a financial analyst, you review a variety of financial data, such as price/ earnings ratios and dividend yields, to guide investment recommendations. How might statistics be used to help you make buy, sell, or hold recommendations for individual stocks?
By comparing statistical information for an individual stock with information about stock market sector averages, you can draw conclusions about whether the stock is overvalued or undervalued. Statistics is used for both "technical analysis" (which considers the trading patterns of stocks) and "quantitative analysis" (which studies economic or company-specific data that might be expected to affect the price or perceived value of a stock).
-
You are a brand manager for a retailer and wish to gain a better understanding of the association between promotional activities and sales. How might statistics be used to help you obtain this information and use it to establish future marketing strategies for your brand?
Electronic scanners at retail checkout counters and online retailer records can provide sales data and statistical summaries on promotional activities such as discount pricing and the use of in-store displays or e-commerce websites. Statistics can be used to model these data to discover which product features appeal to particular market segments and to predict market share for different marketing strategies.
-
As a production manager for a manufacturer, you wish to improve the overall quality of your product by deciding when to make adjustments to the production process, for example, increasing or decreasing the speed of a machine. How might statistics be used to help you make those decisions?
Statistical quality control charts can be used to monitor the output of the production process. Samples from previous runs can be used to determine when the process is "in control." Ongoing samples allow you to monitor when the process goes out of control, so that you can make the adjustments necessary to bring it back into control.
-
As an economist, one of your responsibilities is providing forecasts about some aspect of the economy, for example, the inflation rate. How might statistics be used to estimate those forecasts optimally?
Statistical information on various economic indicators can be entered into computerized forecasting models (also determined using statistical methods) to predict inflation rates. Examples of such indicators include the producer price index, the unemployment rate, and manufacturing capacity utilization.
-
As general manager of a baseball team with limited financial resources, you'd like to obtain strong, yet undervalued players. How might statistics help you to do this?
A wealth of statistical information on baseball player performance is available, and objective analysis of these data can reveal information on those players most likely to add value to the team (in terms of winning games) relative to a player's cost. This field of statistics even has its own name, sabermetrics.
I.2 Learning Statistics
-
What is this book about?
This book is about the application of statistical methods, primarily regression analysis and modeling, to enhance decision-making. Regression analysis is by far the most used statistical methodology in real-world applications. Furthermore, many other statistical techniques are variants or extensions of regression analysis, so once you have a firm foundation in this methodology, you can approach these other techniques without too much additional difficulty. In this book we show you how to apply and interpret regression models, rather than deriving results and formulas (there is no calculus in the book).
-
Why are non-math major students required to study statistics?
In many aspects of modern life, we have to make decisions based on incomplete information (e.g., health, climate, economics, business). This book will help you to understand, analyze, and interpret such data in order to make informed decisions in the face of uncertainty. Statistical theory allows a rigorous, quantifiable appraisal of this uncertainty.
-
How is the book organized?
Chapter 1 reviews the essential details of an introductory statistics course necessary for use in later chapters. Chapter 2 covers the simple linear regression model for analyzing the linear association between two variables (a "response" and a "predictor"). Chapter 3 extends the methods of Chapter 3 to multiple linear regression where there can be more than one predictor variable. Chapters 4 and 5 provide guidance on building regression models, including transforming variables, using interactions, incorporating qualitative information, and diagnosing problems. Chapter 6 (www.wiley.com/go/pardoe/AppliedRegressionModeling3e) contains three case studies that apply the linear regression modeling techniques considered in this book to examples on real estate prices, vehicle...
