Fast inference in convolutional neural networks based on sequential three-way decisions

Abstract

A novel image recognition algorithm based on sequential three-way decisions is introduced to speed up the inference in a convolutional neural network. In contrast to the majority of existing studies, our approach does not require a special procedure to train a neural network, and thus it can be used with arbitrary architectures including pre-trained convolutional nets. Each image is associated with a sequence of features extracted at different layers of the neural network. Features from earlier layers stand for coarse-grained image representation. Fine-grained representations include embeddings from one of later layers. Confidence scores of classifiers representing the input image at each granularity level are computed in order to populate a set of unlikely classes with low confidence scores. The thresholds for these scores are chosen by using the step-up multiple testing procedure. The categories from this set are not considered at the next levels with finer granularity. The algorithm selecting the granularity levels and thresholds for each level is trained on a small sample. An experimental study for several datasets and neural architectures demonstrated that the proposed approach reduces the running time by up to 40% with a controllable decrease in accuracy.