An Efficient RGB-D Scene Recognition Method Based on Multi-Information Fusion

Abstract

RGB-D scene recognition is a challenging problem due to lack of RGB-D datasets and inefficient RGB-D fusion. In this paper, we exploit several factors that affect the RGB-D scene recognition performance, including the representations of the RGB-D data and objects in the scene. We propose an effective multi-information fusion method composed of two modules: a revised detection-based method, and a multi-feature fusion based classifier. The revised detection-based method leverage the auxiliary RGB data. And the multi-feature fusion based classifier select the optimal feature configuration for RGB-D data description. The proposed method is validated on two publicly available datasets: the SUN RGB-D dataset, and the NYU Depth v2 dataset. The obtained results show that the proposed fusion method is effective and is comparable with the state-of-the-art method. Furthermore, the proposed framework contains much less parameters than the state-of-the-art model and thus requires much less time for training. The code and the fine-tuned model parameters are available at: https://github.com/zhangbin28/MulInfo_RGBDScene .