Correlation-Constrained Mutual Information Maximization for Interpretable Multi-Layered Neural Networks

Abstract

The present paper aims to propose a new type of mutual information maximization method to produce humanly interpretable networks, using information on correlation between inputs and targets for creating more easily interpretable connection weights. Multi-layered neural networks have been extensively used in many application fields due to their strong prediction performance, but the neural networks have the serious shortcoming of difficulty in interpreting their inference mechanism. Thus, there is an urgent need to develop a method to produce interpretable results, particularly humanly interpretable ones. For interpretation, we have so far introduced mutual information maximization to simplify complex networks and disentangle connection weights for easy interpretation. The final connection weights were simple enough for interpretation, but the simplified results could not necessarily be accepted by humans, or at least human experts. To make the interpretation more easily understood by humans, we try here to modify our mutual information maximization to produce final connection weights by taking into account correlation coefficients between inputs and targets in the training data set. This means that the final important inputs should be selected by considering direct correlations between inputs and targets, and more notably, the important ones should be chosen among those inputs with a strong correlation with the corresponding targets. This makes it possible to relate the final results to the actual correlations between inputs and targets widely accepted in many fields. The method was applied to two data sets. One was the well-known glass data set, while the other was the real eye-tracking data set. In both cases, the new method could improve generalization performance, and in addition, for the eye-tracking data set, it was found that the results could be more naturally interpreted.