Visual-based obstacle detection and tracking, and conflict detection for small UAS sense and avoid

Abstract

This paper proposes an original approach for visual-based obstacle detection and tracking, and conflict detection, suitable to endow small Unmanned Aircraft Systems with non-cooperative Sense and Avoid capabilities. Specifically, it is designed to detect and track an uncooperative flying object (intruder), and to establish whether or not it represents a collision threat. It is based on three main algorithmic steps, each aided by the use of ownship navigation data. First, visual detection is carried out using two Deep Learning based neural networks (operating above and below the horizon, respectively) followed by local image analysis to improve the accuracy in the detected intruder position on the image plane. Second, tentative track generation and firm tracking are executed exploiting local association, multi-temporal frame differencing, and Kalman filtering. Finally, conflict detection is applied to each firm track based on the estimate of line of sight and line of sight rate in stabilized coordinates. An experimental dataset, which reproduces realistic low-altitude encounter geometries with two customized quadcopters, is collected to assess proposed approach performance. The two vehicles, equipped with high-resolution color cameras, are flown at slightly different altitudes so that the intruder is located above and below the horizon in the two respective image subsets. The proposed approach allows the target to be declared at relatively long range and to be tracked with a line of sight rate accuracy of the order of tenths of degrees per second, which is effective for conflict detection.