Wavelet Neural Networks

Preface xiii

1 Machine Learning and Financial Engineering 1

Financial Engineering 2

Financial Engineering and Related Research Areas 3

Functions of Financial Engineering 5

Applications of Machine Learning in Finance 6

From Neural to Wavelet Networks 8

Wavelet Analysis 8

Extending the Fourier Transform: The Wavelet Analysis Paradigm 10

Neural Networks 17

Wavelet Neural Networks 19

Applications of Wavelet Neural Networks in Financial Engineering, Chaos, and Classification 21

Building Wavelet Networks 23

Variable Selection 23

Model Selection 24

Model Adequacy Testing 25

Book Outline 25

References 27

2 Neural Networks 35

Parallel Processing 36

Processing Units 37

Activation Status and Activation Rules 37

Connectivity Model 39

Perceptron 41

The Approximation Theorem 42

The Delta Rule 42

Backpropagation Neural Networks 44

Multilayer Feedforward Networks 44

The Generalized Delta Rule 45

Backpropagation in Practice 49

Training with Backpropagation 51

Network Paralysis 54

Local Minima 54

Nonunique Solutions 56

Configuration Reference 56

Conclusions 59

References 59

3 Wavelet Neural Networks 61

Wavelet Neural Networks for Multivariate Process Modeling 62

Structure of a Wavelet Neural Network 62

Initialization of the Parameters of the Wavelet Network 64

Training a Wavelet Network with Backpropagation 69

Stopping Conditions for Training 72

Evaluating the Initialization Methods 73

Conclusions 77

References 78

4 Model Selection: Selecting the Architecture of the Network 81

The Usual Practice 82

Early Stopping 82

Regularization 83

Pruning 84

Minimum Prediction Risk 86

Estimating the Prediction Risk Using Information Criteria 87

Estimating the Prediction Risk Using Sampling Techniques 89

Bootstrapping 91

Cross-Validation 94

Model Selection Without Training 95

Evaluating the Model Selection Algorithm 97

Case 1: Sinusoid and Noise with Decreasing Variance 98

Case 2: Sum of Sinusoids and Cauchy Noise 100

Adaptive Networks and Online Synthesis 103

Conclusions 104

References 105

5 Variable Selection: Determining the Explanatory Variables 107

Existing Algorithms 108

Sensitivity Criteria 110

Model Fitness Criteria 112

Algorithm for Selecting the Significant Variables 114

Resampling Methods for the Estimation of Empirical Distributions 116

Evaluating the Variable Significance Criteria 117

Case 1: Sinusoid and Noise with Decreasing Variance 117

Case 2: Sum of Sinusoids and Cauchy Noise 120

Conclusions 123

References 123

6 Model Adequacy: Determining a Network's Future Performance 125

Testing the residuals 126

Testing for Serial Correlation in the Residuals 127

Evaluation Criteria for the Prediction Ability of the Wavelet Network 129

Measuring the Accuracy of the Predictions 129

Scatter Plots 131

Linear Regression Between Forecasts and Targets 132

Measuring the Ability to Predict the Change in Direction 136

Two Simulated Cases 137

Case 1: Sinusoid and Noise with Decreasing Variance 137

Case 2: Sum of Sinusoids and Cauchy Noise 142

Classification 146

Assumptions and Objectives of Discriminant Analysis 146

Validation of the Discriminant Function 148

Evaluating the Classification Ability of a Wavelet Network 150

Case 3: Classification Example on Bankruptcy 153

Conclusions 156

References 156

7 Modeling Uncertainty: From Point Estimates to Prediction Intervals 159

The Usual Practice 160

Confidence and Prediction Intervals 161

Constructing Confidence Intervals 164

The Bagging Method 164

The Balancing Method 165

Constructing Prediction Intervals 166

The Bagging Method 167

The Balancing Method 168

Evaluating the Methods for Constructing Confidence and Prediction Intervals 168

Conclusions 170

References 171

8 Modeling Financial Temperature Derivatives 173

Weather Derivatives 174

Pricing and Modeling Methods 175

Data Description and Preprocessing 176

Data Examination 176

Model for the Daily Average Temperature: Gaussian Ornstein-Uhlenbeck Process with Lags and Time-Varying Mean Reversion 179

Estimation Using Wavelet Networks 183

Variable Selection 183

Model Selection 187

Initialization and Training 187

Confidence and Prediction Intervals 189

Out-of-Sample Comparison 189

Conclusions 191

References 192

9 Modeling Financial Wind Derivatives 197

Modeling the Daily Average Wind Speed 199

Linear ARMA Model 202

Wavelet Networks for Wind Speed Modeling 206

Variable Selection 206

Model Selection 209

Initialization and Training 209

Model Adequacy 209

Speed of Mean Reversion and Seasonal Variance 211

Forecasting Daily Average Wind Speeds 212

Conclusions 215

References 216

10 Predicting Chaotic Time Series 219

Mackey-Glass Equation 220

Model Selection 221

Initialization and Training 221

Model Adequacy 222

Predicting the Evolution of the Chaotic Mackey-Glass Time Series 225

Confidence and Prediction Intervals 226

Conclusions 228

References 229

11 Classification of Breast Cancer Cases 231

Data 232

Part A: Classification of Breast Cancer 232

Model Selection 232

Initialization and Training 233

Classification 233

Part B: Cross-Validation in Breast Cancer Classification in Wisconsin 235

Variable Selection 235

Model Selection 237

Initialization and Training 238

Classification Power of the Full and Reduced Models 238

Part C: Classification of Breast Cancer (Continued) 241

Classification 241

Conclusions 243

References 244

Index 245