Foreword by Ravi Bapna xix
Foreword by Gareth James xxi
Preface to the Second R Edition xxiii
Acknowledgments xxvi
Part I Preliminaries
Chapter 1 Introduction 3
1.1 What Is Business Analytics? 3
1.2 What Is Machine Learning? 5
1.3 Machine Learning, AI, and Related Terms 5
1.4 Big Data 7
1.5 Data Science 8
1.6 Why Are There So Many Different Methods? 8
1.7 Terminology and Notation 9
1.8 Road Maps to This Book 11
Order of Topics 13
Chapter 2 Overview of the Machine Learning Process 17
2.1 Introduction 17
2.2 Core Ideas in Machine Learning 18
Classification 18
Prediction 18
Association Rules and Recommendation Systems 18
Predictive Analytics 19
Data Reduction and Dimension Reduction 19
Data Exploration and Visualization 19
Supervised and Unsupervised Learning 20
2.3 The Steps in a Machine Learning Project 21
2.4 Preliminary Steps 23
Organization of Data 23
Predicting Home Values in the West Roxbury Neighborhood 23
Loading and Looking at the Data in R 24
Sampling from a Database 26
Oversampling Rare Events in Classification Tasks 27
Preprocessing and Cleaning the Data 28
2.5 Predictive Power and Overfitting 35
Overfitting 36
Creating and Using Data Partitions 38
2.6 Building a Predictive Model 41
Modeling Process 41
2.7 Using R for Machine Learning on a Local Machine 46
2.8 Automating Machine Learning Solutions 47
Predicting Power Generator Failure 48
Uber’s Michelangelo 50
2.9 Ethical Practice in Machine Learning 52
Machine Learning Software: The State of the Market (by Herb Edelstein) 53
Problems 57
Part II Data Exploration and Dimension Reduction
Chapter 3 Data Visualization 63
3.1 Uses of Data Visualization 63
Base R or ggplot? 65
3.2 Data Examples 65
Example 1: Boston Housing Data 65
Example 2: Ridership on Amtrak Trains 67
3.3 Basic Charts: Bar Charts, Line Charts, and Scatter Plots 67
Distribution Plots: Boxplots and Histograms 70
Heatmaps: Visualizing Correlations and Missing Values 73
3.4 Multidimensional Visualization 75
Adding Variables: Color, Size, Shape, Multiple Panels, and Animation 76
Manipulations: Rescaling, Aggregation and Hierarchies, Zooming, Filtering 79
Reference: Trend Lines and Labels 83
Scaling Up to Large Datasets 85
Multivariate Plot: Parallel Coordinates Plot 85
Interactive Visualization 88
3.5 Specialized Visualizations 91
Visualizing Networked Data 91
Visualizing Hierarchical Data: Treemaps 93
Visualizing Geographical Data: Map Charts 95
3.6 Major Visualizations and Operations, by Machine Learning Goal 97
Prediction 97
Classification 97
Time Series Forecasting 97
Unsupervised Learning 98
Problems 99
Chapter 4 Dimension Reduction 101
4.1 Introduction 101
4.2 Curse of Dimensionality 102
4.3 Practical Considerations 102
Example 1: House Prices in Boston 103
4.4 Data Summaries 103
Summary Statistics 104
Aggregation and Pivot Tables 104
4.5 Correlation Analysis 107
4.6 Reducing the Number of Categories in Categorical Variables 109
4.7 Converting a Categorical Variable to a Numerical Variable 111
4.8 Principal Component Analysis 111
Example 2: Breakfast Cereals 111
Principal Components 116
Normalizing the Data 117
Using Principal Components for Classification and Prediction 120
4.9 Dimension Reduction Using Regression Models 121
4.10 Dimension Reduction Using Classification and Regression Trees 121
Problems 123
Part III Performance Evaluation
Chapter 5 Evaluating Predictive Performance 129
5.1 Introduction 130
5.2 Evaluating Predictive Performance 130
Naive Benchmark: The Average 131
Prediction Accuracy Measures 131
Comparing Training and Holdout Performance 133
Cumulative Gains and Lift Charts 133
5.3 Judging Classifier Performance 136
Benchmark: The Naive Rule 136
Class Separation 136
The Confusion (Classification) Matrix 137
Using the Holdout Data 138
Accuracy Measures 139
Propensities and Threshold for Classification 139
Performance in Case of Unequal Importance of Classes 143
Asymmetric Misclassification Costs 146
Generalization to More Than Two Classes 149
5.4 Judging Ranking Performance 150
Cumulative Gains and Lift Charts for Binary Data 150
Decile-wise Lift Charts 153
Beyond Two Classes 154
Gains and Lift Charts Incorporating Costs and Benefits 154
Cumulative Gains as a Function of Threshold 155
5.5 Oversampling 156
Creating an Over-sampled Training Set 158
Evaluating Model Performance Using a Non-oversampled Holdout Set 159
Evaluating Model Performance If Only Oversampled Holdout Set Exists 159
Problems 162
Part IV Prediction and Classification Methods
Chapter 6 Multiple Linear Regression 167
6.1 Introduction 167
6.2 Explanatory vs. Predictive Modeling 168
6.3 Estimating the Regression Equation and Prediction 170
Example: Predicting the Price of Used Toyota Corolla Cars 171
Cross-validation and caret 175
6.4 Variable Selection in Linear Regression 176
Reducing the Number of Predictors 176
How to Reduce the Number of Predictors 178
Regularization (Shrinkage Models) 183
Problems 188
Chapter 7 k-Nearest Neighbors (kNN) 193
7.1 The k-NN Classifier (Categorical Outcome) 193
Determining Neighbors 194
Classification Rule 194
Example: Riding Mowers 195
Choosing k 196
Weighted k-NN 199
Setting the Cutoff Value 200
k-NN with More Than Two Classes 201
Converting Categorical Variables to Binary Dummies 201
7.2 k-NN for a Numerical Outcome 201
7.3 Advantages and Shortcomings of k-NN Algorithms 204
Problems 205
Chapter 8 The Naive Bayes Classifier 207
8.1 Introduction 207
Threshold Probability Method 208
Conditional Probability 208
Example 1: Predicting Fraudulent Financial Reporting 208
8.2 Applying the Full (Exact) Bayesian Classifier 209
Using the “Assign to the Most Probable Class” Method 210
Using the Threshold Probability Method 210
Practical Difficulty with the Complete (Exact) Bayes Procedure 210
8.3 Solution: Naive Bayes 211
The Naive Bayes Assumption of Conditional Independence 212
Using the Threshold Probability Method 212
Example 2: Predicting Fraudulent Financial Reports, Two Predictors 213
Example 3: Predicting Delayed Flights 214
Working with Continuous Predictors 218
8.4 Advantages and Shortcomings of the Naive Bayes Classifier 220
Problems 223
Chapter 9 Classification and Regression Trees 225
9.1 Introduction 226
Tree Structure 227
Decision Rules 227
Classifying a New Record 227
9.2 Classification Trees 228
Recursive Partitioning 228
Example 1: Riding Mowers 228
Measures of Impurity 231
9.3 Evaluating the Performance of a Classification Tree 235
Example 2: Acceptance of Personal Loan 236
9.4 Avoiding Overfitting 239
Stopping Tree Growth 242
Pruning the Tree 243
Best-Pruned Tree 245
9.5 Classification Rules from Trees 247
9.6 Classification Trees for More Than Two Classes 248
9.7 Regression Trees 249
Prediction 250
Measuring Impurity 250
Evaluating Performance 250
9.8 Advantages and Weaknesses of a Tree 250
9.9 Improving Prediction: Random Forests and Boosted Trees 252
Random Forests 252
Boosted Trees 254
Problems 257
Chapter 10 Logistic Regression 261
10.1 Introduction 261
10.2 The Logistic Regression Model 263
10.3 Example: Acceptance of Personal Loan 264
Model with a Single Predictor 265
Estimating the Logistic Model from Data: Computing Parameter Estimates 267
Interpreting Results in Terms of Odds (for a Profiling Goal) 270
10.4 Evaluating Classification Performance 271
10.5 Variable Selection 273
10.6 Logistic Regression for Multi-Class Classification 274
Ordinal Classes 275
Nominal Classes 276
10.7 Example of Complete Analysis: Predicting Delayed Flights 277
Data Preprocessing 282
Model-Fitting and Estimation 282
Model Interpretation 282
Model Performance 284
Variable Selection 285
Problems 289
Chapter 11 Neural Nets 293
11.1 Introduction 293
11.2 Concept and Structure of a Neural Network 294
11.3 Fitting a Network to Data 295
Example 1: Tiny Dataset 295
Computing Output of Nodes 296
Preprocessing the Data 299
Training the Model 300
Example 2: Classifying Accident Severity 304
Avoiding Overfitting 305
Using the Output for Prediction and Classification 305
11.4 Required User Input 307
11.5 Exploring the Relationship Between Predictors and Outcome 308
11.6 Deep Learning 309
Convolutional Neural Networks (CNNs) 310
Local Feature Map 311
A Hierarchy of Features 311
The Learning Process 312
Unsupervised Learning 312
Example: Classification of Fashion Images 313
Conclusion 320
11.7 Advantages and Weaknesses of Neural Networks 320
Problems 322
Chapter 12 Discriminant Analysis 325
12.1 Introduction 325
Example 1: Riding Mowers 326
Example 2: Personal Loan Acceptance 327
12.2 Distance of a Record from a Class 327
12.3 Fisher’s Linear Classification Functions 329
12.4 Classification Performance of Discriminant Analysis 333
12.5 Prior Probabilities 334
12.6 Unequal Misclassification Costs 334
12.7 Classifying More Than Two Classes 336
Example 3: Medical Dispatch to Accident Scenes 336
12.8 Advantages and Weaknesses 339
Problems 341
Chapter 13 Generating, Comparing, and Combining Multiple Models 345
13.1 Ensembles 346
Why Ensembles Can Improve Predictive Power 346
Simple Averaging or Voting 348
Bagging 349
Boosting 349
Bagging and Boosting in R 349
Stacking 350
Advantages and Weaknesses of Ensembles 351
13.2 Automated Machine Learning (AutoML) 352
AutoML: Explore and Clean Data 352
AutoML: Determine Machine Learning Task 353
AutoML: Choose Features and Machine Learning Methods 354
AutoML: Evaluate Model Performance 354
AutoML: Model Deployment 356
Advantages and Weaknesses of Automated Machine Learning 357
13.3 Explaining Model Predictions 358
13.4 Summary 360
Problems 362
345
Part V Intervention and User Feedback
Chapter 14 Interventions: Experiments, Uplift Models, and Reinforcement Learning 367
14.1 A/B Testing 368
Example: Testing a New Feature in a Photo Sharing App 369
The Statistical Test for Comparing Two Groups (T-Test) 370
Multiple Treatment Groups: A/B/n Tests 372
Multiple A/B Tests and the Danger of Multiple Testing 372
14.2 Uplift (Persuasion) Modeling 373
Gathering the Data 374
A Simple Model 376
Modeling Individual Uplift 376
Computing Uplift with R 378
Using the Results of an Uplift Model 378
14.3 Reinforcement Learning 380
Explore-Exploit: Multi-armed Bandits 380
Example of Using a Contextual Multi-Arm Bandit for Movie Recommendations 382
Markov Decision Process (MDP) 383
14.4 Summary 388
Problems 390
Part VI Mining Relationships Among Records
Chapter 15 Association Rules and Collaborative Filtering 393
15.1 Association Rules 394
Discovering Association Rules in Transaction Databases 394
Example 1: Synthetic Data on Purchases of Phone Faceplates 394
Generating Candidate Rules 395
The Apriori Algorithm 397
Selecting Strong Rules 397
Data Format 399
The Process of Rule Selection 400
Interpreting the Results 401
Rules and Chance 403
Example 2: Rules for Similar Book Purchases 405
15.2 Collaborative Filtering 407
Data Type and Format 407
Example 3: Netflix Prize Contest 408
User-Based Collaborative Filtering: “People Like You” 409
Item-Based Collaborative Filtering 411
Evaluating Performance 412
Example 4: Predicting Movie Ratings with MovieLens Data 413
Advantages and Weaknesses of Collaborative Filtering 416
Collaborative Filtering vs. Association Rules 417
15.3 Summary 419
Problems 421
Chapter 16 Cluster Analysis 425
16.1 Introduction 426
Example: Public Utilities 427
16.2 Measuring Distance Between Two Records 429
Euclidean Distance 429
Normalizing Numerical Variables 430
Other Distance Measures for Numerical Data 432
Distance Measures for Categorical Data 433
Distance Measures for Mixed Data 434
16.3 Measuring Distance Between Two Clusters 434
Minimum Distance 434
Maximum Distance 435
Average Distance 435
Centroid Distance 435
16.4 Hierarchical (Agglomerative) Clustering 437
Single Linkage 437
Complete Linkage 438
Average Linkage 438
Centroid Linkage 438
Ward’s Method 438
Dendrograms: Displaying Clustering Process and Results 439
Validating Clusters 441
Limitations of Hierarchical Clustering 443
16.5 Non-Hierarchical Clustering: The k-Means Algorithm 444
Choosing the Number of Clusters (k) 445
Problems 450
Part VII Forecasting Time Series
Chapter 17 Handling Time Series 455
17.1 Introduction 455
17.2 Descriptive vs. Predictive Modeling 457
17.3 Popular Forecasting Methods in Business 457
Problems 466
Chapter 18 Regression-Based Forecasting 469
18.1 A Model with Trend 469
Linear Trend 469
Exponential Trend 473
Polynomial Trend 474
Problems 489
Chapter 19 Smoothing and Deep Learning Methods for Forecasting 499
19.1 Smoothing Methods: Introduction 500
19.2 Moving Average 500
Centered Moving Average for Visualization 500
Trailing Moving Average for Forecasting 501
Choosing Window Width (w) 504
Problems 516
Part VIII Data Analytics
Chapter 20 Social Network Analytics 527
20.1 Introduction 527
20.2 Directed vs. Undirected Networks 529
20.3 Visualizing and Analyzing Networks 530
Plot Layout 530
Edge List 533
Adjacency Matrix 533
Using Network Data in Classification and Prediction 534
Problems 548
Chapter 21 Text Mining 549
21.1 Introduction 549
21.2 The Tabular Representation of Text 550
21.3 Bag-of-Words vs. Meaning Extraction at Document Level 551
Problems 570
Chapter 22 Responsible Data Science 573
22.1 Introduction 573
22.2 Unintentional Harm 574
22.3 Legal Considerations 576
22.4 Principles of Responsible Data Science 577
Non-maleficence 578
Fairness 578
Transparency 579
Accountability 580
Data Privacy and Security 580
Problems 599
Part IX Cases
Chapter 23 Cases 603
23.1 Charles Book Club 603
The Book Industry 603
Database Marketing at Charles 604
Machine Learning Techniques 606
Assignment 608
23.2 German Credit 610
Background 610
Data 610
Assignment 614
Index 647
