Dual-resolution transformer combined with multi-layer separable convolution fusion network for real-time semantic segmentation

Abstract

Environmental perception is crucial for unmanned mobile platforms such as autonomous vehicles and robots. Precise and fast semantic segmentation of the surrounding scene is a key task to enhance this capability. Existing real-time semantic segmentation networks are typically based on convolutional neural networks (CNNs), which have achieved good results, but they still lack control over global context features. In recent years, the Transformer architecture has achieved significant success in capturing global context, which is beneficial for improving segmentation accuracy. However, Transformers tend to ignore local connections, and their computational complexity makes real-time segmentation challenging. We propose a lightweight real-time semantic segmentation network called DTMC-Net, which combines the advantages of CNNs and Transformers. We design a special residual convolution module called the Lightweight Multi-layer Separable Convolution Attention module (LMSCA) to reduce the parameter count and perform multi-scale feature fusion to capture local features effectively. We introduce the Simple Dual-Resolution Transformer (SDR Transformer) that utilizes lightweight attention mechanisms and residual feed forward networks to capture and maintain features, with multiple bilateral fusions between two branches to exchange information. The proposed Anti-artifact Aggregation Pyramid Pooling Module (AAPPM) optimizes the upsampling process, refines features, and performs multi-scale feature fusion again. DTMC-Net only contains 4.2M parameters and achieves good performance on multiple public datasets with different scenarios.