BiSeNet V3: Bilateral segmentation network with coordinate attention for real-time semantic segmentation

Abstract

For the semantic segmentation task, spatial information and the receptive field are indispensable. For semantic segmentation to be practically applicable, it must have real-time inference speed. However, most of today’s methods almost choose to compromise the spatial resolution and low-level detail information, which leads to a significant decrease in accuracy. In this paper, we propose a new architecture based on Bilateral Segmentation Network (BiSeNet) called BiSeNet V3. It introduces a new feature refinement module to optimize the feature map and a feature fusion module to combine the features efficiently. An attention mechanism is introduced to assist the model in capturing contextual information. We also use edge detection to enhance features for boundaries. Extensive experiments on the Cityscapes dataset show that our proposed approach achieves an excellent performance between segmentation accuracy and inference speed. Specifically, for a 768 × 1536 input, BiSeNet V3 achieved 79.0% mIoU on the Cityscapes test set with a speed of 93.8 FPS on an NVIDIA GTX 1080Ti. For a 720 × 960 input, BiSeNet V3 achieved 76.6% mIoU on the CamVid dataset with a speed of 147.6 FPS on an NVIDIA GTX 1080Ti. The result is significantly better than the other methods.