DRNet: Dual-stage refinement network with boundary inference for RGB-D semantic segmentation of indoor scenes

Abstract

Semantic segmentation is a dense pixel prediction task, and its accuracy depends on the extraction of long-range contextual knowledge and refinement of segmentation boundaries. Most segmentation methods are based on feature extraction using a convolutional neural network, and layer-by-layer sampling and fusion are applied to solve inherent problems such as chaotic boundaries and scattered objects. Owing to the limited receptive field and loss of details during downsampling, the segmentation results may be unsatisfactory. To address existing shortcomings, we propose a dual-stage refinement network (DRNet) for semantic segmentation. In the first stage, we adopt an efficient spatiotemporal representation learning framework called UniFormer. We also use a novel boundary extractor and initial segmentation map generator to obtain rough segmentation results. In the second stage, we use the rough segmentation map and extracted boundary information in a graph reasoning module that restores the class boundary features while completing global modeling and local information inference. Benefiting from the acquisition of long-range dependencies between image pixels, contextual information promotes the distinction of pixel categories. In addition, edge information can increase the interclass distinguishability and refine the segmentation boundaries. Results from extensive experiments demonstrate that the proposed DRNet outperforms state-of-the-art semantic segmentation methods. The codes and results are available at: https://github.com/EnquanYang2022/DRNet.