Feature screening using signal-to-noise ratios

Abstract

We present an approach for identifying important input features in multi-layer perceptron feedforward artificial neural networks (ANN) trained via backpropagation. Specifically, we propose a new saliency measure and demonstrate its use in a methodology that holds the promise of potentially identifying and removing noisy input features in a single training run. First, we propose a signal-to-noise ratio (SNR) saliency measure, which determines the saliency of a feature by comparing it to that of an injected noise feature. Next, we propose a feature screening method that utilizes the SNR saliency measure to select a parsimonious set of salient features. The SNR screening method is demonstrated using Fisher's Iris problem. Confidence in the SNR saliency measure and screening method are bolstered by comparisons to Setiono and Liu's neural network feature selector and a principal component analysis (PCA) approach on three real-world applications: the University of Wisconsin Breast Cancer Diagnosis problem, the US Congressional voting records problem, and the Pima Indians Diabetes problem.