Applied Predictive Analytics

Introduction xxi Chapter 1 Overview of Predictive Analytics 1 What Is Analytics? 3 What Is Predictive Analytics? 3 Supervised vs. Unsupervised Learning 5 Parametric vs. Non-Parametric Models 6 Business Intelligence 6 Predictive Analytics vs. Business Intelligence 8 Do Predictive Models Just State the Obvious? 9 Similarities between Business Intelligence and Predictive Analytics 9 Predictive Analytics vs. Statistics 10 Statistics and Analytics 11 Predictive Analytics and Statistics Contrasted 12 Predictive Analytics vs. Data Mining 13 Who Uses Predictive Analytics? 13 Challenges in Using Predictive Analytics 14 Obstacles in Management 14 Obstacles with Data 14 Obstacles with Modeling 15 Obstacles in Deployment 16 What Educational Background Is Needed to Become a Predictive Modeler? 16 Chapter 2 Setting Up the Problem 19 Predictive Analytics Processing Steps: CRISP-DM 19 Business Understanding 21 The Three-Legged Stool 22 Business Objectives 23 Defining Data for Predictive Modeling 25 Defining the Columns as Measures 26 Defining the Unit of Analysis 27 Which Unit of Analysis? 28 Defining the Target Variable 29 Temporal Considerations for Target Variable 31 Defining Measures of Success for Predictive Models 32 Success Criteria for Classifi cation 32 Success Criteria for Estimation 33 Other Customized Success Criteria 33 Doing Predictive Modeling Out of Order 34 Building Models First 34 Early Model Deployment 35 Case Study: Recovering Lapsed Donors 35 Overview 36 Business Objectives 36 Data for the Competition 36 The Target Variables 36 Modeling Objectives 37 Model Selection and Evaluation Criteria 38 Model Deployment 39 Case Study: Fraud Detection 39 Overview 39 Business Objectives 39 Data for the Project 40 The Target Variables 40 Modeling Objectives 41 Model Selection and Evaluation Criteria 41 Model Deployment 41 Summary 42 Chapter 3 Data Understanding 43 What the Data Looks Like 44 Single Variable Summaries 44 Mean 45 Standard Deviation 45 The Normal Distribution 45 Uniform Distribution 46 Applying Simple Statistics in Data Understanding 47 Skewness 49 Kurtosis 51 Rank-Ordered Statistics 52 Categorical Variable Assessment 55 Data Visualization in One Dimension 58 Histograms 59 Multiple Variable Summaries 64 Hidden Value in Variable Interactions: Simpson's Paradox 64 The Combinatorial Explosion of Interactions 65 Correlations 66 Spurious Correlations 66 Back to Correlations 67 Crosstabs 68 Data Visualization, Two or Higher Dimensions 69 Scatterplots 69 Anscombe's Quartet 71 Scatterplot Matrices 75 Overlaying the Target Variable in Summary 76 Scatterplots in More Than Two Dimensions 78 The Value of Statistical Signifi cance 80 Pulling It All Together into a Data Audit 81 Summary 82 Chapter 4 Data Preparation 83 Variable Cleaning 84 Incorrect Values 84 Consistency in Data Formats 85 Outliers 85 Multidimensional Outliers 89 Missing Values 90 Fixing Missing Data 91 Feature Creation 98 Simple Variable Transformations 98 Fixing Skew 99 Binning Continuous Variables 103 Numeric Variable Scaling 104 Nominal Variable Transformation 107 Ordinal Variable Transformations 108 Date and Time Variable Features 109 ZIP Code Features 110 Which Version of a Variable Is Best? 110 Multidimensional Features 112 Variable Selection Prior to Modeling 117 Sampling 123 Example: Why Normalization Matters for K-Means Clustering 139 Summary 143 Chapter 5 Itemsets and Association Rules 145 Terminology 146 Condition 147 Left-Hand-Side, Antecedent(s) 148 Right-Hand-Side, Consequent, Output, Conclusion 148 Rule (Item Set) 148 Support 149 Antecedent Support 149 Confi dence, Accuracy 150 Lift 150 Parameter Settings 151 How the Data Is Organized 151 Standard Predictive Modeling Data Format 151 Transactional Format 152 Measures of Interesting Rules 154 Deploying Association Rules 156 Variable Selection 157 Interaction Variable Creation 157 Problems with Association Rules 158 Redundant Rules 158 Too Many Rules 158 Too Few Rules 159 Building Classification Rules from Association Rules 159 Summary 161 Chapter 6 Descriptive Modeling 163 Data Preparation Issues with Descriptive Modeling 164 Principal Component Analysis 165 The PCA Algorithm 165 Applying PCA to New Data 169 PCA for Data Interpretation 171 Additional Considerations before Using PCA 172 The Effect of Variable Magnitude on PCA Models 174 Clustering Algorithms 177 The K-Means Algorithm 178 Data Preparation for K-Means 183 Selecting the Number of Clusters 185 The Kohonen SOM Algorithm 192 Visualizing Kohonen Maps 194 Similarities with K-Means 196 Summary 197 Chapter 7 Interpreting Descriptive Models 199 Standard Cluster Model Interpretation 199 Problems with Interpretation Methods 202 Identifying Key Variables in Forming Cluster Models 203 Cluster Prototypes 209 Cluster Outliers 210 Summary 212 Chapter 8 Predictive Modeling 213 Decision Trees 214 The Decision Tree Landscape 215 Building Decision Trees 218 Decision Tree Splitting Metrics 221 Decision Tree Knobs and Options 222 Reweighting Records: Priors 224 Reweighting Records: Misclassifi cation Costs 224 Other Practical Considerations for Decision Trees 229 Logistic Regression 230 Interpreting Logistic Regression Models 233 Other Practical Considerations for Logistic Regression 235 Neural Networks 240 Building Blocks: The Neuron 242 Neural Network Training 244 The Flexibility of Neural Networks 247 Neural Network Settings 249 Neural Network Pruning 251 Interpreting Neural Networks 252 Neural Network Decision Boundaries 253 Other Practical Considerations for Neural Networks 253 K-Nearest Neighbor 254 The k-NN Learning Algorithm 254 Distance Metrics for k-NN 258 Other Practical Considerations for k-NN 259 Naïve Bayes 264 Bayes' Theorem 264 The Naïve Bayes Classifier 268 Interpreting Naïve Bayes Classifi ers 268 Other Practical Considerations for Naïve Bayes 269 Regression Models 270 Linear Regression 271 Linear Regression Assumptions 274 Variable Selection in Linear Regression 276 Interpreting Linear Regression Models 278 Using Linear Regression for Classification 279 Other Regression Algorithms 280 Summary 281 Chapter 9 Assessing Predictive Models 283 Batch Approach to Model Assessment 284 Percent Correct Classifi cation 284 Rank-Ordered Approach to Model Assessment 293 Assessing Regression Models 301 Summary 304 Chapter 10 Model Ensembles 307 Motivation for Ensembles 307 The Wisdom of Crowds 308 Bias Variance Tradeoff 309 Bagging 311 Boosting 316 Improvements to Bagging and Boosting 320 Random Forests 320 Stochastic Gradient Boosting 321 Heterogeneous Ensembles 321 Model Ensembles and Occam's Razor 323 Interpreting Model Ensembles 323 Summary 326 Chapter 11 Text Mining 327 Motivation for Text Mining 328 A Predictive Modeling Approach to Text Mining 329 Structured vs. Unstructured Data 329 Why Text Mining Is Hard 330 Text Mining Applications 332 Data Sources for Text Mining 333 Data Preparation Steps 333 POS Tagging 333 Tokens 336 Stop Word and Punctuation Filters 336 Character Length and Number Filters 337 Stemming 337 Dictionaries 338 The Sentiment Polarity Movie Data Set 339 Text Mining Features 340 Term Frequency 341 Inverse Document Frequency 344 TF-IDF 344 Cosine Similarity 346 Multi-Word Features: N-Grams 346 Reducing Keyword Features 347 Grouping Terms 347 Modeling with Text Mining Features 347 Regular Expressions 349 Uses of Regular Expressions in Text Mining 351 Summary 352 Chapter 12 Model Deployment 353 General Deployment Considerations 354 Deployment Steps 355 Summary 375 Chapter 13 Case Studies 377 Survey Analysis Case Study: Overview 377 Business Understanding: Defining the Problem 378 Data Understanding 380 Data Preparation 381 Modeling 385 Deployment: "What-If" Analysis 391 Revisit Models 392 Deployment 401 Summary and Conclusions 401 Help Desk Case Study 402 Data Understanding: Defining the Data 403 Data Preparation 403 Modeling 405 Revisit Business Understanding 407 Deployment 409 Summary and Conclusions 411 Index 413