Optimal strategies for wide-area small object detection using deep learning: Practices from a global flying aircraft dataset

Abstract

Deep learning (DL) has made tremendous strides in object detection in remote sensing (RS). Still, due to the limitation of small objects' nature (e.g., size, shape, and texture), their detecting performance is less than half that of medium/large objects in closed-set conditions (all test classes are known in the training phase). In the real world, small-object detection (SOD) over wide areas is far more challenging: models trained with limited positives cannot robustly handle unknown classes (i.e., negatives or backgrounds with similar features to positives), namely, the open-set recognition (OSR) problem. To better perform deep-learning-based SOD, we built a global large-scale training set—flying aircraft (FlyingAC), including ∼3.0 × 105 positives and 2.5 × 105 negatives. We then proposed a series of optimal strategies involving set construction, model training, set volume, model structure, and inference mode. Evaluated based on the FlyingAC, the proposed optimal strategies can mitigate the OSR problem and enhance the inference efficiency. Specifically, we found that (i) training the DL models together with positives and negatives is the most effective way to alleviate the OSR problem (improving the F1 score from 11 % to >90 %). (ii) Optimum training sets volumes exist for DL-based SOD. For the FlyingAC, the optimum volume is 2.0 × 105 positives and 1.5 × 105 negatives. (iii) Optimizing the DL model, such as FPN, attention mechanism, and loss function, can improve the SOD precision. Still, the improvement from the model level is much smaller than the optimization from the dataset level. And (iv) For wide-area detection, the candidate-based inference mode we designed can reduce the inference time to one-fifth and the number of false positives by ∼50 % compared to the commonly used sliding-window inference mode.