An iterative feature reduction algorithm for probabilistic neural networks

Abstract

The issue of noise and redundant feature variables in classification problems has been well addressed in statistical stepwise discriminant analysis. However, no feature reduction algorithm designed specifically for Probabilistic Neural Network (PNN) is found in literature. In this study, we develop an ad hoc iterative feature (variable) reduction algorithm for basic PNNs to identify noise and redundant variables. A basic PNN applying the same smoothing factor to all variables does not distinguish variables adding little or no predictive power to the model from others. The proposed iterative approach utilizes a weighted PNN with one smoothing factor for each variable in the feature reduction stage. Once a subset of variables is selected, a basic PNN is developed on the chosen variables for future applications. Computational experiments on five data sets obtained from publicly available sources indicate that the basic PNN with variables chosen by the ad hoc approach outperforms the PNN using variables selected by two benchmark methods, back-propagation neural network and stepwise linear discriminant analysis.