DeepScene: Scene classification via convolutional neural network with spatial pyramid pooling

Abstract

Dissimilar to object classification, scene classification needs to consider not only the components that exist in the image but also their corresponding distribution. The greatest challenge of scene classification, especially indoor scene classification, is that many classes share the same representative components whereas the degree of similarity can be low within the same class. Some images have no clear indication that they belong to a particular class. In view of this, we propose a DeepScene model that leverages Convolutional Neural Network as the base architecture. As color cues are important for scene classification, two solutions are proposed to convert grayscale scene images to RGB images, which are replication and deep neural network based style transfer for colorization. To address the challenge of objects with varying sizes and positions in the scene, Spatial Pyramid Pooling is incorporated into the Convolutional Neural Network. The Spatial Pyramid Pooling performs multi-level pooling to enable the multi-size training of the model for improved scale and translational invariance. Ensemble learning is then adopted to boost the overall performance in scene classification. The proposed DeepScene model outshines the state-of-art methods with accuracy of 98.1% on Event-8, 95.6% on Scene-15 and 71.0% on MIT-67.