Sensitivity Analysis for Neural Networks

Cover -- Contents -- 1 Introduction to Neural Networks -- 1.1 Properties of Neural Networks -- 1.2 Neural Network Learning -- 1.2.1 Supervised Learning -- 1.2.2 Unsupervised Learning -- 1.3 Perceptron -- 1.4 Adaline and Least Mean Square Algorithm -- 1.5 Multilayer Perceptron and Backpropagation Algorithm -- 1.5.1 Output Layer Learning -- 1.5.2 Hidden Layer Learning -- 1.6 Radial Basis Function Networks -- 1.7 Support Vector Machines -- 2 Principles of Sensitivity Analysis -- 2.1 Perturbations in Neural Networks -- 2.2 Neural Network Sensitivity Analysis -- 2.3 Fundamental Methods of Sensitivity Analysis -- 2.3.1 Geometrical Approach -- 2.3.2 Statistical Approach -- 2.4 Summary -- 3 Hyper-Rectangle Model -- 3.1 Hyper-Rectangle Model for Input Space of MLP -- 3.2 Sensitivity Measure of MLP -- 3.3 Discussion -- 4 Sensitivity Analysis with Parameterized Activation Function -- 4.1 Parameterized Antisymmetric Squashing Function -- 4.2 Sensitivity Measure -- 4.3 Summary -- 5 Localized Generalization Error Model -- 5.1 Introduction -- 5.2 The Localized Generalization Error Model -- 5.2.1 The Q-Neighborhood and Q-Union -- 5.2.2 The Localized Generalization Error Bound -- 5.2.3 Stochastic Sensitivity Measure for RBFNN -- 5.2.4 Characteristics of the Error Bound -- 5.2.5 Comparing Two Classifiers Using the Error Bound -- 5.3 Architecture Selection Using the Error Bound -- 5.3.1 Parameters for MC SG -- 5.3.2 RBFNN Architecture Selection Algorithm for MC SG -- 5.3.3 A Heuristic Method to Reduce the Computational Time for MC SG -- 5.4 Summary -- 6 Critical Vector Learning for RBF Networks -- 6.1 RelatedWork -- 6.2 Construction of RBF Networks with Sensitivity Analysis -- 6.2.1 RBF Classifiers8217; Sensitivity to the Kernel Function Centers -- 6.2.2 Orthogonal Least Square Transform -- 6.2.3 Critical Vector Selection -- 6.3 Summary -- 7 Sensitivity Analysis of Prior Knowledge -- 7.1 KBANNs -- 7.2 Inductive Bias -- 7.3 Sensitivity Analysis and Measures -- 7.3.1 Output-Pattern Sensitivity -- 7.3.2 Output-Weight Sensitivity -- 7.3.3 Output-H Sensitivity -- 7.3.4 Euclidean Distance -- 7.4 Promoter Recognition -- 7.4.1 Data and Initial Domain Theory -- 7.4.2 Experimental Methodology -- 7.5 Discussion and Conclusion -- 8 Applications -- 8.1 Input Dimension Reduction -- 8.1.1 Sensitivity Matrix -- 8.1.2 Criteria for Pruning Inputs -- 8.2 Network Optimization -- 8.3 Selective Learning -- 8.4 Hardware Robustness -- 8.5 Measure of Nonlinearity -- 8.6 Parameter Tuning for Neocognitron -- 8.6.1 Receptive Field -- 8.6.2 Selectivity -- 8.6.3 Sensitivity Analysis of the Neocognitron -- Bibliography.