Abstract
Abstract. Indoor scenes have the characteristics of abundant semantic categories, illumination changes, occlusions and overlaps among objects, which poses great challenges for indoor semantic segmentation. Therefore, we in this paper develop a method based on higher-order Markov random field model for indoor semantic segmentation from RGB-D images. Instead of directly using RGB-D images, we first train and perform RefineNet model only using RGB information for generating the high-level semantic information. Then, the spatial location relationship from depth channel and the spectral information from color channels are integrated as a prior for a marker-controlled watershed algorithm to obtain the robust and accurate visual homogenous regions. Finally, higher-order Markov random field model encodes the short-range context among the adjacent pixels and the long-range context within each visual homogenous region for refining the semantic segmentations. To evaluate the effectiveness and robustness of the proposed method, experiments were conducted on the public SUN RGB-D dataset. Experimental results indicate that compared with using RGB information alone, the proposed method remarkably improves the semantic segmentation results, especially at object boundaries.
Highlights
Semantic segmentation is a fundamental problem in computer vision, which decomposes a scene into meaningful parts and assigns semantic labels to them (Wolf et al, 2015)
To address the issues raised from the state-of-the-art of the semantic segmentation for indoor scenes, we develop a method based on higher-order Markov random field model for indoor semantic segmentation from RGB-D images
We develop a method based on higher-order Markov random field (MRF) model, which combines the highlevel semantic information derived from RefineNet and the lowlevel visual information captured from a marker-controlled watershed algorithm, for indoor semantic segmentation from RGB-D images
Summary
Semantic segmentation is a fundamental problem in computer vision, which decomposes a scene into meaningful parts and assigns semantic labels to them (Wolf et al, 2015). Müller and Behnke (2014) conducted conditional random filed, into which color, depth and 3D scene features were incorporated, for semantic annotation of RGB-D images. These conventional methods usually consist of segmentation, feature extraction and classification and their final results depend on the results of each stage (Husain et al, 2016). Occlusions and overlaps among objects in indoor scenes, the spatial location relationship from depth channel and the spectral information from color channels are integrated as prior information for a marker-controlled watershed algorithm to derive the robust and accurate visual homogenous regions, which will encode the low-level visual features for complementarily reconstructing the detailed boundaries.
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have