Feature Fusion with Multi-Task Attention Networks for Semantic Segmentation from VHR Remote Sensing Images

Abstract

Semantic segmentation from very high-resolution (VHR) remote sensing images is of great significance in a broad spectrum of applications. In recent years, deep learning techniques have been used to accomplish relevant tasks. Many of such attempts start with selecting only three image channels for fusion, followed by establishing a segmentation model, making it tough to exploit the information of all individual image channels. Consequently, fusing multi-modal data for improving segmentation performance has received considerable research attention. In this article, we propose a novel approach enabling effective feature fusion for better semantic segmentation from VHR remote sensing images. Based on the encoder-decoder model, our approach’s encoder comprises two branch networks simultaneously extracting features from heterogeneous image data. In particular, we introduce multi-task attention networks (MTANs) in the encoder to fuse feature maps, enabling both branch networks to complement each other through the task. Experiments demonstrated that our approach outperformed several benchmark models on two image datasets, proving its promise in semantic segmentation from VHR remote sensing images.s given in this document.