DFFNet: An IoT-perceptive dual feature fusion network for general real-time semantic segmentation

Abstract

Semantic segmentation is a valuable yet challenging research in the Internet of Things (IoT), especially for some real-time and resource-constrained applications. Recently we witness a strong tendency of fusing multi-level features or multi-scale context information to achieve promising segmentation performance. However, we find that existing literature has at least one of the following issues: 1) relying on more resource-consuming feature extraction operations, e.g., standard convolution with large kernels, for multiple information fusion; 2) seldom considering that how to narrow the semantic gap between multi-layer features, leading to sub-optimal performance. To tackle these issues, we propose a novel IoT-perceptive Dual Feature Fusion Network (DFFNet) for semantic segmentation, which aims to leverage multi-level features and multi-scale context information in an efficient yet effective manner. Specifically, the multi-level feature fusion module (MFFM), which enhances the semantic consistency between multi-level features by two attention refinement blocks, is proposed to exploit multi-layer features for jointly learning spatial and semantic information with small overheads. Moreover, a multi-scale component termed as lightweight semantic pyramid module (LSPM) is presented to improve the efficiency of context encoding by depthwise factorized convolution operations. Extensive experimental results on benchmarks have demonstrated that DFFNet achieves better performance than existing advanced methods.