Layer factor analysis in convolutional neural networks for explainability

Abstract

Explanatory methods that focus on the analysis of the features encoded by Convolutional Neural Networks (CNNs) are of great interest, since they help to understand the underlying process hidden behind the black-box nature of these models. However, to explain the knowledge gathered in a given layer, they must decide which of the numerous filters to study, further assuming that each of them corresponds to a single feature. This, coupled with the redundancy of information, makes it difficult to ensure that the relevant characteristics are being analyzed. The above represents an important challenge and defines the aim and scope of our proposal. In this paper we present a novel method, named Explainable Layer Factor Analysis for CNNs (ELFA-CNNs), which models and describes with quality convolutional layers relying on factor analysis. Regarding contributions, ELFA obtains the essential underlying features, together with their correlation with the original filters, providing an accurate and well-founded summary. Through the factorial parameters we gain insights about the information learned, the connections between channels, and the redundancy of the layer, among others. To provide visual explanations in a similarly way to other methods, two additional proposals are made: a) Essential Feature Attribution Maps (EFAM) and b) intrinsic features inversion. The results prove the effectiveness of the developed general methods. They are evaluated in different CNNs (VGG-16, ResNet-50, and DeepLabv3+) on generic datasets (CIFAR-10, imagenette, and CamVid). We demonstrate that convolutional layers adequately fit a factorial model thanks to the new metrics presented for factor and fitting residuals (D1, D>, and Res, derive from covariance matrices). Moreover, knowledge about the deep image representations and the learning process is acquired, as well as reliable heat maps highlighting regions where essential features are located. This study effectively provides an explainable approach that can be applied to different CNNs and over different datasets.