Propositional Rules Generated at the Top Layers of a CNN

Abstract

So far, many rule extraction techniques have been proposed to explain the classifications of shallow Multi Layer Perceptrons (MLPs), but very few methods have been introduced for Convolutional Neural Networks (CNNs). To fill this gap, this work presents a new technique applied to a CNN architecture including two convolutional layers. This neural network is trained with the MNIST dataset, representing images of digits. Rule extraction is performed at the first fully connected layer, by means of the Discretized Interpretable Multi Layer Perceptron (DIMLP). This transparent MLP architecture allows us to generate symbolic rules, by precisely locating axis-parallel hyperplanes. The antecedents of the extracted rules represent responses of convolutional filters that makes it possible to determine for each rule the covered samples. Hence, we can visualize the centroid of each rule, which gives us some insight into how the network works. This represents a first step towards the explanation of CNN responses, since the final explanation would be obtained in a further processing by generating propositional rules with respect to the input layer. In the experiments we illustrate a generated ruleset with its characteristics in terms of accuracy, complexity and fidelity, which is the degree of matching between CNN classifications and rules classifications. Overall, rules reach very high fidelity. Finally, several examples of rules are visualized and discussed.