RELAXNet: Residual efficient learning and attention expected fusion network for real-time semantic segmentation

Abstract

As a dense prediction problem, semantic segmentation consumes extensive memory and computational resources. However, the application of semantic segmentation requires the model to perform real-time analyses in portable devices, thus it is crucial to seek a trade-off between segmentation accuracy and inference speed. In this paper, we propose a lightweight semantic segmentation method based on attention mechanism to address this problem. First, we use novel Efficient Bottleneck Residual (EBR) Module and Efficient Asymmetric Bottleneck Residual (EABR) Module to extract both local and contextual information,which adopt a well-designed combination of depth-wise convolution, dilated convolution and factorized convolution, with channel shuffle to boost information interaction. Second, we introduce attention mechanism into skip connection between the encoder and decoder to promote reasonable fusion of high-level and low-level features, which furtherly enhance the accuracy. With only 1.9 M parameters, our model obtains 74.8% mIoU and 64 FPS running speed on Cityscapes dataset and 71.2% mIoU and 79 FPS running speed on Camvid dataset. Experiments demonstrate that our model achieves competitive results in terms of segmentation accuracy and running speed while controlling parameters.