A Novel Semantic Segmentation Algorithm for RGB-D Images Based on Non-Symmetry and Anti-Packing Pattern Representation Model

Abstract

With the rapid development of deep learning technology, the accuracy of image semantic segmentation tasks has been greatly improved. However, indoor RGB-D semantic segmentation remains a challenging problem because of the complexity of indoor environments. The emergence of depth sensors makes depth information gradually used to improve the effect of semantic segmentation. The splicing of weights such as between the RGB features and the depth features which are used as the input features of the neural network can effectively improve the accuracy of the indoor semantic segmentation tasks. Most previous researches focused on improving the performance of semantic segmentation by adjusting the structure of convolutional neural network. These researches have either added attention mechanism or performed data augmentation on input features, but they didn’t make full use of the boundary information and the texture information of the original RGB image. In this paper, we propose a semantic segmentation algorithm for RGB-D images based on Non-symmetry and Anti-packing pattern representation Model (NAM). The core idea of the proposed algorithm is to take the channel-wise concatenation of pre-segmentation labels provided by the traditional hierarchical image segmentation and RGB-D features as the input of the neural network so as to guide the semantic segmentation tasks. The extensive experiments are conducted on the popular indoor RGB-D semantic segmentation datasets. When compared with the state-of-art algorithms, the experimental results presented in this paper show that our proposed method has improved the performance of image semantic segmentation networks on several popular neural network architectures.