Interpretation and knowledge discovery from the multilayer perceptron network: Opening the black box

Abstract

This paper interprets the outputs from the multilayer perceptron (MLP) network by finding the input data features at the input layer of the network which activate the hidden layer feature detectors. This leads directly to the deduction of the significant data inputs, the inputs that the network actually uses to perform the input/output mapping for a classification task, and the discovery of the most significant of these data inputs. The analysis presents a method for providing explanations for the network outputs and for representing the knowledge learned by the network in the form of significant input data relationships. During network development the explanation facilities and data relationships can be used for network validation and verification, and after development, for rule induction and data mining where this method provides a potential tool for knowledge discovery in databases (KDD).