A study of the difference between partial derivative and stochastic neural network sensitivity analysis for applications in supervised pattern classification problems

Abstract

This work provides a brief development roadmap of the neural network sensitivity analysis, from 1960's to now on. The two main streams of the sensitivity measures: partial derivative and stochastic sensitivity measures are compared. The partial derivative sensitivity (PD-SM) finds the rate of change of the network output with respect to parameter changes, while the stochastic sensitivity (ST-SM) finds the magnitudes of the output perturbations between the original training samples and the perturbed samples, in statistical sense. Their computational complexities are compared. Furthermore, how to evaluate multiple parameters of the neural network with or without correlation are explored too. In addition, the differences of them in the application of supervised pattern classification problems are also discussed. The evaluations are based on three major applications of sensitivity analysis in supervised pattern classification problems: feature selection, sample selection and neural network generalization assessment. ST-SM and PD-SM of the RBFNN are used for investigations.