A Lightweight Adaptive RoI Extraction Network for Precise Aerial Image Instance Segmentation

Abstract

Bounding boxes have been widely implemented into aerial object detection for their simplicity. They perform instance-level locations with the coordinates and orientation for each target. However, the defects such as coarse edge information impede semantic interpretation in Earth observation. Besides, in terms of the aerial imaging instruments, it is essential to recognize the exterior appearance and contour of the objects. In this article, we propose a novel aerial instance segmentation method termed the adaptive region of interest (RoI) extraction network (ARE-Net), which bridges the gap of accurately delineating instances under the complex background of aerial images. To exert instance segmentation under the proprietary property, e.g., complex background and densely distributed instances, of aerial images, RoIs are pooled from multilevel feature maps and integral region proposals. On this basis, global attention RoI extractor (GA-RoIE) and perceptual RoI extractor (PRoIE) are, respectively, introduced for detection branch and mask branch to perform adaptive RoI extraction for aerial images. Meanwhile, to reconcile the probability distribution regional distribution of pixelwise prediction in aerial images, we present the adaptive compound loss function to improve the integrating degree of the predicted binary mask to the ground-truth mask. In addition, we adopt RegNetx with deformable convolution to optimize ARE-Net and name it as R-ARE-Net. Despite implementing pixelwise prediction, comprehensive experiments on iSAID and NWPU VHR-10 instance segmentation datasets still have verified the effectiveness and efficiency of ARE-Net and R-ARE-Net. Experimental results indicate that our proposed methods receive the highest AP value (38.0% AP on iSAID and 64.2% AP on NWPU VHR-10 instance segmentation dataset) and the lowest floating-point operations (FLOPs) and parameters’ consumption (~46% reduced FLOPs and 61.5% reduced parameters than SCNet) among the mainstream methods. Besides, the false alarms, missing segmentations, poorly predicted masks, and undersegmentations that appeared in the mainstream methods can be avoided to some extend for R-ARE-Net.