Instance-Aware Spatial-Frequency Feature Fusion Detector for Oriented Object Detection in Remote-Sensing Images

Abstract

In recent years, fusing multi-type features poses great potential for oriented object detection (OOD) in remote sensing images (RSIs). Due to the inexplicit operation of modeling orientation variations, convolutional neural networks (CNNs) are difficult to perceive objects under different transformations (angles and scales). In this paper, we propose a novel instance-aware spatial-frequency feature fusion detector (SFFD) for oriented object detection in remote sensing images. First, a layer-wise frequency-domain analysis (L-FDA) module is built along with CNN layers to extract frequency features. Getting rid of the constrains such as horizontal rectangular kernel in CNNs, our L-FDA possesses outstanding ability of locating mutational signals from frequency space. These mutational signals record the scale and angle information of the oriented instances in images. Subsequently, CNN and frequency features are sent into RoI Pooling layer to obtain multi-type instance-level RoI features. Moreover, the proposed instance-aware cross feature fusion (CFF) module explores the interaction between these diverse features which provides an explicit indicator to compensate the orientation information ignored by instance-level CNN features. Finally, our SFFD unifies the proposed L-FDA module and CFF module into the detection network to localize oriented instances in RSIs. We compare our method with many state-of-the-art methods on DOTA, HRSC2016, and NWPU VHR-10 datasets. Experimental results verify the validity of modeling instance-level object relations from frequency-domain and CNNs for OOD.