Modulation Learning on Fourier-Domain for Road Extraction From Remote Sensing Images

Abstract

Extraction road from remote sensing (RS) images is a challenging topic because of the inhomogeneous intensity, non-consistent contrast, and very cluttered background of satellite images. Most previous approaches, relying on convolutions or self-attention, are built on local operation or global modeling on spatial domain, but are difficult to capture weak and continuous road objects. The spectral representation of road image features and modulation learning on it provide a novel long-range dependent and fine-grained feature representation mechanism. Based on it, we propose a novel road extraction network on RS images, called an adaptive Fourier filtered U-shaped network (AFU-Net) in this letter, which relies on modulation learning on the Fourier-domain. The AFU-Net is composed of modulation learner (MoL) basic blocks and follows the pipeline of classical U-Net model. The basic MoL block includes a global modulation learner (GML) block for global spectral modulation learning and an attentive modulation learner (AML) block which contains two parallel layers, i.e. phase-modulated filter (PMF) and magnitude-modulated filter (MMF), for fine-grained spectral modulation on the Fourier spectrum. The experiments on two public datasets, such as Massachusetts Roads and DeepGlobe Road Datasets have shown outstanding performance of AFU-Net on the metrics of accuracy, precision, recall and mIoU etc.