Abstract

Abstract. Indoor scenes have the characteristics of abundant semantic categories, illumination changes, occlusions and overlaps among objects, which poses great challenges for indoor semantic segmentation. Therefore, we in this paper develop a method based on higher-order Markov random field model for indoor semantic segmentation from RGB-D images. Instead of directly using RGB-D images, we first train and perform RefineNet model only using RGB information for generating the high-level semantic information. Then, the spatial location relationship from depth channel and the spectral information from color channels are integrated as a prior for a marker-controlled watershed algorithm to obtain the robust and accurate visual homogenous regions. Finally, higher-order Markov random field model encodes the short-range context among the adjacent pixels and the long-range context within each visual homogenous region for refining the semantic segmentations. To evaluate the effectiveness and robustness of the proposed method, experiments were conducted on the public SUN RGB-D dataset. Experimental results indicate that compared with using RGB information alone, the proposed method remarkably improves the semantic segmentation results, especially at object boundaries.

Highlights

  • Semantic segmentation is a fundamental problem in computer vision, which decomposes a scene into meaningful parts and assigns semantic labels to them (Wolf et al, 2015)

  • To address the issues raised from the state-of-the-art of the semantic segmentation for indoor scenes, we develop a method based on higher-order Markov random field model for indoor semantic segmentation from RGB-D images

  • We develop a method based on higher-order Markov random field (MRF) model, which combines the highlevel semantic information derived from RefineNet and the lowlevel visual information captured from a marker-controlled watershed algorithm, for indoor semantic segmentation from RGB-D images

Read more

Summary

INTRODUCTION

Semantic segmentation is a fundamental problem in computer vision, which decomposes a scene into meaningful parts and assigns semantic labels to them (Wolf et al, 2015). Müller and Behnke (2014) conducted conditional random filed, into which color, depth and 3D scene features were incorporated, for semantic annotation of RGB-D images. These conventional methods usually consist of segmentation, feature extraction and classification and their final results depend on the results of each stage (Husain et al, 2016). Occlusions and overlaps among objects in indoor scenes, the spatial location relationship from depth channel and the spectral information from color channels are integrated as prior information for a marker-controlled watershed algorithm to derive the robust and accurate visual homogenous regions, which will encode the low-level visual features for complementarily reconstructing the detailed boundaries.

METHODOLOGY
Initial semantic segmentation using RefineNet
Region-level label consistency based on higher-order MRF model
EXPERIMENTATION AND ANALYSIS
Experimental data and evaluation criteria
Experimental analysis
CONCLUSION
Findings
Background
Full Text
Published version (Free)

Talk to us

Join us for a 30 min session where you can share your feedback and ask us any queries you have

Schedule a call