Pixel-Adaptive Field-of-View for Remote Sensing Image Segmentation

Abstract

Mineral segmentation of satellite imagery is crucial to mining surveying and monitoring. Conventional deep segmentation networks extract features at every position with a fixed field-of-view. Nevertheless, the rich content in large mineral scenes, which causes dramatically different local characteristics across regions, may require features with spatially varying field-of-view to achieve accurate segmentation. In light of this, we propose a novel Pixel-Adaptive Field-of-View (PA-FoV) module to adjust the field-of-view of a given feature map in a pixel-wise manner. Specifically, it refines the features at each position by a weighted aggregation of the outputs from atrous convolutions with different dilation rates adaptively depending on the position-specific content. The module works in a plug-and-play manner and can be flexibly inserted into any arbitrary backbone network or segmentation head, to boost feature representation and in turn improve the result of segmentation. Moreover, in order to mitigate the scarcity of labeled data, we further establish a benchmark remote sensing mineral dataset, dubbed RSMI, to facilitate research in this field. Extensive experiments show a simple addition of our PA-FoV module provides solid improvements on top of strong baselines, achieving state-of-the-art performance.