A survey on indoor RGB-D semantic segmentation: from hand-crafted features to deep convolutional neural networks

Abstract

Semantic segmentation is one of the most important tasks in the field of computer vision. It is the main step towards scene understanding. With the advent of RGB-Depth sensors, such as Microsoft Kinect, nowadays RGB-Depth images are easily available. This has changed the landscape of some tasks such as semantic segmentation. As the depth images are independent of illumination, the combination of depth and RGB images can improve the quality of semantic labeling. The related research has been divided into two main categories, based on the usage of hand-crafted features and deep learning. Although the state-of-the-art results are mainly achieved by deep learning methods, traditional methods have also been at the center of attention for some years and lots of valuable work have been done in that category. As the field of semantic segmentation is very broad, in this survey, a comprehensive analysis has been carried out on RGB-Depth semantic segmentation methods, their challenges and contributions, available RGB-Depth datasets, metrics of evaluation, state-of-the-art results, and promising directions of the field.