Machine Learning in Geomechanics 1

Preface ix
Ioannis STEFANOU and Felix DARVE

Chapter 1 Overview of Machine Learning in Geomechanics 1
Ioannis STEFANOU

1.1. What exactly is machine learning? 1

1.2. Classification of ML methods 7

1.2.1. Supervised versus unsupervised ml 7

1.2.2. Batch versus online ml 9

1.2.3. Instance-based versus model-based ml 10

1.3. ML and geomechanics 12

1.4. Libraries for ml 16

1.5. Bias in ML and limitations 16

1.6. What to expect from these volumes? 19

1.7. Acknowledgments 20

1.8. References 20

Chapter 2 Introduction to Regression Methods 31
Filippo MASI

2.1. Introduction 32

2.2. Linear regression 34

2.2.1. Example 38

2.3. Gradient descent 41

2.3.1. Batch GD 46

2.3.2. Stochastic GD 48

2.3.3. Mini-batch GD 50

2.4. Data preprocessing and model validation 54

2.4.1. Feature scaling 54

2.4.2. Test and validation of a model 56

2.5. Nonlinear regression 58

2.5.1. End-to-end example 59

2.6. Regularization techniques 66

2.6.1. Over- and under-determined systems 66

2.6.2. Regularized regression 69

2.7. Challenges in generalization and extrapolation 72

2.7.1. Interpretable models and where to find them 74

2.8. Bayesian regression 81

2.8.1. Linear Bayesian regression 82

2.8.2. GP regression 85

2.9. Conclusions 89

2.10. References 90

Chapter 3 Unsupervised Learning: Basic Concepts and Application to Particle Dynamics 93
Noel JAKSE

3.1. Introduction 93

3.2. Basic concepts 95

3.2.1. Representation of the data: Feature extraction and selection 95

3.2.2. Distance and similarity metrics 97

3.3. Unsupervised learning techniques 99

3.3.1. Clustering 99

3.3.2. Dimensionality reduction 102

3.4. Application to particle dynamics 104

3.4.1. Topological description of local structures 106

3.4.2. Clustering local environments during nucleation 109

3.5. Conclusion 111

3.6. Acknowledgements 112

3.7. References 112

Chapter 4 Classification Techniques in Machine Learning 117
Noel JAKSE

4.1. Introduction 117

4.2. Classification techniques 119

4.2.1. General considerations 119

4.2.2. Typical workflow for classification 120

4.2.3. Evaluation metrics 122

4.2.4. Standard classification algorithms 125

4.3. AL in classification 137

4.3.1. Application of classification 141

4.4. Conclusion 142

4.5. Acknowledgments 143

4.6. References 143

Chapter 5 Artificial Neural Networks: Learning the Optimum Statistical Model from Data 145
Filippo GATTI

5.1 Why PyTorch? 146

5.2. Introduction to sampling theory 150

5.2.1. Statistical models and maximum likelihood estimator 156

5.2.2. On the MLE optimization problem 173

5.2.3. The Fisher information: geometric interpretation 176

5.2.4. The Fisher information: statistical interpretation 177

5.2.5. The principle of maximum entropy (MaxEnt) 184

5.3. Optimizing a neural network 188

5.3.1. First-order gradient descent for empirical loss minimization 195

5.3.2. Second-order gradient descent methods 202

5.3.3. The role of BatchNorm 209

5.3.4. Stochastic gradient descent 212

5.3.5. Beyond SGD: the role of “momentum” 215

5.3.6. Beyond classical momentum SGD: the Nesterov algorithm 218

5.3.7. Optimizing with adaptive learning rates 220

5.4. References 232

List of Authors 237

Index 239

Summary of Volume 2 243