Classification and feature selection with fused conditionally dependent binary valued features

Abstract

In this paper, the Bayesian Data Reduction Algorithm (BDRA) is compared to several neural networks to demonstrate classification performance and feature selection for fused binary valued features, where the statistical dependency (i.e., correlation or redundancy) between the relevant features of each class is varied. The BDRA uses the probability of error, conditioned on the training data, and a "greedy" approach (similar to a backward sequential feature search) for reducing irrelevant features from the data. Results are shown by plotting the probability of error as a function of the conditional probability between adjacent relevant features, where the number of relevant features is varied. In general, it is demonstrated that the performance difference between the BDRA and the neural networks depends on the statistical dependency between the features.