LSNet: Real-time attention semantic segmentation network with linear complexity

Abstract

From high-resolution images to get context information remains a challenge in real-time semantic segmentation. Despite recent advances in context information acquisition based on Self-Attention,the Self-Attention module will cause the calculation degree to increase exponentially when facing the semantic segmentation task. In this article, we propose a real-time semantic segmentation network (LSNet) which strikes a balance between segmentation accuracy and inference speed. Firstly, we propose a linear attention module to reduce the computational complexity of Self-Attention from O(n2) to O(n). Secondly, we propose a context information fusion module by using a cross-attention mechanism to fill in a lot of spatial details for high-level semantic information. Finally, we design a real-time semantic segmentation network LSNet to greatly improve the real-time inference speed and ensure the accuracy of network segmentation. To verify the effectiveness of the LSNet network, we conducted a lot of comparative experiments on CamVid, Cityscapes, COCO-Stuff datasets, and the experimental results show that LSNet achieves an excellent trade-off between accuracy and speed on both Cityscapes, CamVid and COCO-Stuff datasets.