Gated-Residual Block for Semantic Segmentation Using RGB-D Data

Abstract

Semantic segmentation is an important technique for scene understanding in the intelligent transportation system. RGB-D data shows great advantages over the unimodal data in this area, and it can be easily obtained from consumer sensors nowadays. How to design effective fusion structures to fuse RGB and depth signals in RGB-D data is a challenging problem. This paper proposes a novel gated-residual block to address this problem. The structure consists of two residual units and one gated fusion unit, the residual unit progressively aggregates modality-specific features from the modality-specific signals and the gate mechanism computes complementary features for them. Based on the gated-residual block, the paper presents the deep multimodal networks, named GRBNet, for RGB-D semantic segmentation. Experiments on ScanNet, Cityscapes and SUN RGB-D datasets verify the effectiveness of the proposed approach and demonstrate that the GRBNet achieved competitive performance.