NEFELI: A deep-learning detection and tracking pipeline for enhancing autonomy in advanced air mobility

Abstract

Efficient detection and accurate collision estimation for non-cooperative aerial vehicles are crucial for the realization of fully autonomous aircraft and Advanced Air Mobility (AAM). This paper introduces NEFELI, a machine learning software utilizing optical sensors to detect and track non-cooperative aerial vehicles. NEFELI's detector employs an enhanced YOLOv5-large model, strengthened with a sliced inference step to enhance detection capabilities for distant, diminutive objects. Furthermore, NEFELI introduces several innovations in its tracking component. A key advancement lies in the creation and utilization of the first large-scale re-identification (Re-ID) dataset of aerial objects. This dataset is used to train the deep learning appearance (Re-ID) model of the tracking module and integrates appearance information into the detection and tracking pipeline, resulting in more robust and reliable tracking performance. Moreover, the tracking model combines the deep learning appearance model with a Kalman Filter-based motion model to address the challenge of precisely tracking distant aerial objects. Notably, an extensive comparative analysis that was conducted showed that NEFELI outperforms state-of-the-art detection and tracking models in terms of Higher Order Tracking Accuracy (HOTA) metric, ID switches, and Association Accuracy (AssA) by a wide margin. A crucial aspect of this work is NEFELI's software architecture design, which enables efficient implementation on a low SWaP (Size, Weight, and Power) edge Graphic Processing Unit (GPU). To further showcase NEFELI's generalization capabilities and edge implementation performance, real-world flight experiments with small UAVs were carried out. The experimental results demonstrate NEFELI's ability to detect and track small UAVs at distances of up to 145 m in real-time speed of 6.7 fps.