A Deep Learning-based Indoor Scene Classification Approach Enhanced with Inter-Object Distance Semantic Features

Abstract

Convolutional Neural Networks (CNNs) have been increasingly applied in visual classification tasks by replacing hand-crafted features with deep features. However, problems such as inter-class similarity and intra-class variation led to the need of obtaining more descriptive features. To accomplish this, a new semantic inter-object relationship approach is proposed, which is based on the distance relationships between recognized objects. This new source of information represents how close or apart objects belonging to two object classes are, which, together with the number of object occurrences, allows to develop a more descriptive semantic feature representation of the scene. To exploit such semantic features, a two-branch CNN architecture based on 1D and 2D convolutional layers, is proposed. Also, an enhancement version, GSF <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> AppV2, of the Global and Semantic Feature Fusion described in [1] is proposed by integrating the new semantic inter-object relationship approach, as well as the aforementioned two-branch CNN architecture. The GSF <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> AppV2 is also composed of a CNN-based global feature branch, where five different CNN-based feature extraction approaches were assessed as global feature extraction modules. Moreover, to combine global and semantic features, two feature fusion approaches are proposed and evaluated: correlation and triple concatenation. GSF <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> AppV2 was evaluated in two benchmark datasets: the SUN RGB-D and NYU Depth V2. State-of-the-art results were achieved on both datasets, showing the effectiveness of the proposed semantic feature approach on the pipeline.