A novel information geometric approach to variable selection in MLP networks

Abstract

In this paper, a novel information geometric-based variable selection criterion for multi-layer perceptron networks is described. It is based on projections of the Riemannian manifold defined by a multi-layer perceptron network on submanifolds defined by multi-layer perceptron networks with reduced input dimension. We show how the divergence between models can be used as a criterion for an efficient search in the space of networks with different inputs. Then, we show how the posterior probabilities of the models can be evaluated to rank the projected models. Finally, we test our algorithm on synthetic and real data, and compare its performances with other methods reported in literature.