Data Mining for Business Analytics

Foreword xvii Preface to the Third Edition xix Preface to the First Edition xxii Acknowledgments xxiv PART I PRELIMINARIES CHAPTER 1 Introduction 3 1.1 What is Business Analytics? 3 1.2 What is Data Mining? 5 1.3 Data Mining and Related Terms 5 1.4 Big Data 6 1.5 Data Science 7 1.6 Why Are There So Many Different Methods? 8 1.7 Terminology and Notation 9 1.8 Road Maps to This Book 11 Order of Topics 12 CHAPTER 2 Overview of the Data Mining Process 14 2.1 Introduction 14 2.2 Core Ideas in Data Mining 15 2.3 The Steps in Data Mining 18 2.4 Preliminary Steps 20 2.5 Predictive Power and Overfitting 26 2.6 Building a Predictive Model with XLMiner 30 2.7 Using Excel for Data Mining 40 2.8 Automating Data Mining Solutions 40 Data Mining Software Tools (by Herb Edelstein) 42 Problems 45 PART II DATA EXPLORATION AND DIMENSION REDUCTION CHAPTER 3 Data Visualization 50 3.1 Uses of Data Visualization 50 3.2 Data Examples 52 Example 1: Boston Housing Data 52 Example 2: Ridership on Amtrak Trains 53 3.3 Basic Charts: Bar Charts, Line Graphs, and Scatter Plots 53 Distribution Plots 54 Heatmaps: Visualizing Correlations and Missing Values 57 3.4 Multi-Dimensional Visualization 58 Adding Variables 59 Manipulations 61 Reference: trend line and labels 64 Scaling up to Large Datasets 65 Multivariate Plot 66 Interactive Visualization 67 3.5 Specialized Visualizations 70 Visualizing Networked Data 70 Visualizing Hierarchical Data: Treemaps 72 Visualizing Geographical Data: Map Charts 73 3.6 Summary: Major Visualizations and Operations, by Data Mining Goal 75 Prediction 75 Classification 75 Time Series Forecasting 75 Unsupervised Learning 76 Problems 77 CHAPTER 4 Dimension Reduction 79 4.1 Introduction 79 4.2 Curse of Dimensionality 80 4.3 Practical Considerations 80 Example 1: House Prices in Boston 80 4.4 Data Summaries 81 4.5 Correlation Analysis 84 4.6 Reducing the Number of Categories in Categorical Variables 85 4.7 Converting A Categorical Variable to A Numerical Variable 86 4.8 Principal Components Analysis 86 Example 2: Breakfast Cereals 87 Principal Components 92 Normalizing the Data 93 Using Principal Components for Classification and Prediction 94 4.9 Dimension Reduction Using Regression Models 96 4.10 Dimension Reduction Using Classification and Regression Trees 96 Problems 97 PART III PERFORMANCE EVALUATION CHAPTER 5 Evaluating Predictive Performance 101 5.1 Introduction 101 5.2 Evaluating Predictive Performance 102 Benchmark: The Average 102 Prediction Accuracy Measures 103 5.3 Judging Classifier Performance 106 Benchmark: The Naive Rule 107 Class Separation 107 The Classification Matrix 107 Using the Validation Data 109 Accuracy Measures 109 Cutoff for Classification 110 Performance in Unequal Importance of Classes 114 Asymmetric Misclassification Costs 116 5.4 Judging Ranking Performance 119 5.5 Oversampling 123 Problems 129 PART IV PREDICTION AND CLASSIFICATION METHODS CHAPTER 6 Multiple Linear Regression 134 6.1 Introduction 134 6.2 Explanatory vs. Predictive Modeling 135 6.3 Estimating the Regression Equation and Prediction 136 Example: Predicting the Price of Used Toyota Corolla Cars 137 6.4 Variable Selection in Linear Regression 141 Reducing the Number of Predictors 141 How to Reduce the Number of Predictors 142 Problems 147 CHAPTER 7 k-Nearest Neighbors (kNN) 151 7.1 The k-NN Classifier (categorical outcome) 151 Determining Neighbors 151 Classification Rule 152 Example: Riding Mowers 152 Choosing k 154 Setting the Cutoff Value 154 7.2 k-NN for a Numerical Response 156 7.3 Advantages and Shortcomings of k-NN Algorithms 158 Problems 160 CHAPTER 8 The Naive Bayes Classifier 162 8.1 Introduction 162 Example 1: Predicting Fraudulent Financial Reporting 163 8.2 Applying the Full (Exact) Bayesian Classifier 164 8.3 Advantages and Shortcomings of the Naive Bayes Classifier 172 Advantages and Shortcomings of the naive Bayes Classifier 172 Problems 176 CHAPTER 9 Classification and Regression Trees 178 9.1 Introduction 178 9.2 Classification Trees 179 Example 1: Riding Mowers 180 9.3 Measures of Impurity 183 9.4 Evaluating the Performance of a Classification Tree 187 Example 2: Acceptance of Personal Loan 188 9.5 Avoiding Overfitting 192 Stopping Tree Growth: CHAID 192 Pruning the Tree 193 9.6 Classification Rules from Trees 198 9.7 Classification Trees for More Than two Classes 198 9.8 Regression Trees 198 Prediction 199 Measuring Impurity 200 Evaluating Performance 200 9.9 Advantages and Weaknesses of a Tree 200 9.10 Improving Prediction: Multiple Trees 202 Problems 205 CHAPTER 10 Logistic Regression 209 10.1 Introduction 209 10.2 The Logistic Regression Model 211 Example: Acceptance of Personal Loan 212 Model with a Single Predictor 214 Estimating the Logistic Model from Data 215 Interpreting Results in Terms of Odds 218 10.3 Evaluating Classification Performance 219 Variable Selection 220 10.4 Example of Complete Analysis: Predicting Delayed Flights 222 Data Preprocessing 224 Model Fitting and Estimation 224 Model Interpretation 226 Model Performance 226 Variable Selection 227 10.5 Appendix: Logistic Regression for Profiling 231 Appendix A: Why Linear Regression Is Problematic for a Categorical Response 231 Appendix B: Evaluating Explanatory Power 233 Appendix C: Logistic Regression for More Than Two Classes 235 Problems 239 CHAPTER 11 Neural Nets 242 11.1 Introduction 242 11.2 Concept and Structure of a Neural Network 243 11.3 Fitting a Network to Data 243 Example 1: Tiny Dataset 244 Computing Output of Nodes 245 Preprocessing the Data 248 Training the Model 248 Example 2: Classifying Accident Severity 253 Avoiding overfitting 254 Using the Output for Prediction and Classification 258 11.4 Required User Input 258 11.5 Exploring the Relationship Between Predictors and Response 259 11.6 Advantages and Weaknesses of Neural Networks 261 Problems 262 CHAPTER 12 Discriminant Analysis 264 12.1 Introduction 264 Example 1: Riding Mowers 265 Example 2: Personal Loan Acceptance 265 12.2 Distance of an Observation from a Class 267 12.3 Fisher s Linear Classification Functions 268 12.4 Classification Performance of Discriminant Analysis 272 12.5 Prior Probabilities 273 12.6 Unequal Misclassification Costs 274 12.7 Classifying More Than Two Classes 274 Example 3: Medical Dispatch to Accident Scenes 274 12.8 Advantages and Weaknesses 277 Problems 279 CHAPTER 13 Combining Methods: Ensembles and Uplift Modeling 282 13.1 Ensembles 282 Why Ensembles Can Improve Predictive Power 283 Simple Averaging 284 Bagging 286 Boosting 286 Advantages and Weaknesses of Ensembles 286 13.2 Uplift (Persuasion) Modeling 287 A-B Testing 287 Uplift 288 Gathering the Data 288 A Simple Model 289 Modeling Individual Uplift 290 Using the Results of an Uplift Model 292 13.3 Summary 292 Problems 293 PART V MINING RELATIONSHIPS AMONG RECORDS CHAPTER 14 Association Rules and Collaborative Filtering 297 14.1 Association Rules 297 Discovering Association Rules in Transaction Databases 298 Example 1: Purchases of Phone Faceplates 298 Generating Candidate Rules 298 The Apriori Algorithm 301 Selecting Strong Rules 301 Data Format 303 The Process of Rule Selection 304 Interpreting the Results 306 Rules and Chance 306 Example 2: Rules for Similar Book Purchases 308 14.2 Collaborative Filtering1 310 Data Type and Format 311 Example 3: Netflix Prize Contest 311 User-Based Collaborative Filtering: People Like You 312 Item-Based Collaborative Filtering 315 Advantages and Weaknesses of Collaborative Filtering 316 Collaborative Filtering vs. Association Rules 316 14.3 Summary 318 Problems 320 CHAPTER 15 Cluster Analysis 324 15.1 Introduction 324 Example: Public Utilities 326 15.2 Measuring Distance Between Two Observations 328 Euclidean Distance 328 Normalizing Numerical Measurements 328 Other Distance Measures for Numerical Data 329 Distance Measures for Categorical Data 331 Distance Measures for Mixed Data 331 15.3 Measuring Distance Between Two Clusters 332 15.4 Hierarchical (Agglomerative) Clustering 334 Single Linkage 335 Complete Linkage 335 Average Linkage 336 Centroid Linkage 336 Dendrograms: Displaying Clustering Process and Results 337 Validating Clusters 339 Limitations of Hierarchical Clustering 340 15.5 Non-hierarchical Clustering: The k-Means Algorithm 341 Initial Partition into k Clusters 342 Problems 346 PART VI FORECASTING TIME SERIES CHAPTER 16 Handling Time Series 351 16.1 Introduction 351 16.2 Descriptive vs. Predictive Modeling 352 16.3 Popular Forecasting Methods in Business 353 Combining Methods 353 16.4 Time Series Components 354 Example: Ridership on Amtrak Trains 354 16.5 Data Partitioning and Performance Evaluation 358 Benchmark Performance: Naive Forecasts 359 Generating Future Forecasts 359 Problems 361 CHAPTER 17 Regression-Based Forecasting 364 17.1 A Model with Trend 364 Linear Trend 364 Exponential Trend 366 Polynomial Trend 369 17.2 A Model with Seasonality 370 17.3 A model with trend and seasonality 371 17.4 Autocorrelation and ARIMA Models 371 Computing Autocorrelation 374 Improving Forecasts by Integrating Autocorrelation Information 376 Evaluating Predictability 380 Problems 382 CHAPTER 18 Smoothing Methods 392 18.1 Introduction 392 18.2 Moving Average 393 Centered Moving Average for Visualization 393 Trailing Moving Average for Forecasting 395 Choosing Window Width (w) 399 18.3 Simple Exponential Smoothing 399 Choosing Smoothing Parameter 400 Relation Between Moving Average and Simple Exponential Smoothing 401 18.4 Advanced Exponential Smoothing 402 Series with a Trend 402 Series with a Trend and Seasonality 403 Series with Seasonality (No Trend) 403 Problems 405 PART VII DATA ANALYTICS CHAPTER 19 Social Network Analytics 415 19.1 Introduction 415 19.2 Directed vs. Undirected Networks 416 19.3 Visualizing and analyzing networks 418 Graph Layout 418 Adjacency List 421 Adjacency Matrix 422 Using Network Data in Classification and Prediction 422 19.4 Social Data Metrics and Taxonomy 423 Node-Level Centrality Metrics 423 Egocentric Network 424 Network Metrics 425 19.5 Using Network Metrics in Prediction and Classification 427 Link Prediction 427 Entity Resolution 427 Collaborative Filtering 428 Advantages and Disadvantages 431 Problems 434 CHAPTER 20 Text Mining 436 20.1 Introduction 436 20.2 The Spreadsheet Representation of Text: Bag-of-Words 437 20.3 Bag-of-Words vs. Meaning Extraction at Document Level 437 20.4 Preprocessing the Text 438 Tokenization 439 Text Reduction 439 Presence/Absence vs. Frequency 440 Term Frequency - Inverse Document Frequency (TF-IDF) 441 From Terms to Concepts: Latent Semantic Indexing 441 Extracting Meaning 441 20.5 Implementing data mining methods 442 20.6 Example: Online Discussions on Autos and Electronics 442 Importing and Labeling the Records 443 Tokenization 444 Text Processing and Reduction 444 Producing a Concept Matrix 444 Labeling the Documents 447 Fitting a Model 447 Prediction 449 20.7 Summary 449 Problems 450 PART VIII CASES CHAPTER 21 Cases 454 21.1 Charles Book Club2 454 21.2 German Credit 463 21.3 Tayko Software Cataloger3 468 21.4 Political Persuasion4 472 21.5 Taxi Cancellations5 475 21.6 Segmenting Consumers of Bath Soap6 477 21.7 Direct-Mail Fundraising 480 21.8 Catalog Cross-Selling7 483 21.9 Predicting Bankruptcy 484 21.10Time Series Case: Forecasting Public Transportation Demand 487 References 489