A Comparison of Gate Detection Algorithms for Autonomous Racing Drones

Abstract

In this paper, several algorithms for gate detection in ADR (autonomous drone racing) are compared based on accuracy and speed. These are YOLOv5 (You Only Look Once), PP-YOLO, Faster R-CNN (Region Based Convolutional Neural Network), U-Net, and Lines (an original algorithm introduced in the paper). Two variants of the first three are trained, one that detects entire gates and another that detects gate corners. Each of these produce around 99% sufficiently accurate gate center predictions in average times ranging from 20 to 60 ms per image, with YOLOv5 being the fastest overall. U-Net and Lines were generally less accurate and slower, with U-Net scoring 88.70% accuracy at 109.90 ms per image, and Lines scoring 34.64% accuracy at 67.60 ms per image. The paper is a novel contribution as it is the first to compare object detection, image segmentation, and classical computer vision methods for this task in a controlled manner. The comparison is performed across the same set of training and test data, on the same hardware, and uses a consistent scoring mechanism, yielding more correct and complete results than one based on a review of studies on the individual algorithms. Variations on the studied algorithms have been commonly used in published work on ADR, but not directly compared [1]–[3]. The original Lines algorithm serves as a representative for algorithms employing classical computer vision methods, leveraging simplicity and domain knowledge in hopes of faster speed. It combines edge and line detection with a simple heuristic to find the boundaries of a gate. The algorithms are compared in their “out of the box” states, without modification or optimization, so that the results are generalizable and valuable for both inexperienced users and experienced users who are prototyping. The algorithms predicted the centers and corners of gates that drones navigate through in a race, where each image contained one gate. They were trained and tested using 9,300 images of real gates from the 2019 AlphaPilot Challenge ADR competition. Accuracy was measured as the distance between the labelled and predicted gate center and corner locations relative to the apparent size of the gate in the image. A threshold was chosen to consider a prediction sufficiently accurate. Speed was measured as the elapsed wall clock time for prediction on powerful desktop hardware, under the assumption that their relative prediction speed would scale to hardware onboard a drone. The results of this study could facilitate future work in ADR by providing information on the performance of various common gate detection methods. ADR itself is exciting to watch, with rewards for top teams continuing to grow, but ADR also serves as a testbed for cutting edge drone hardware and software. Its advancement will ultimately improve drone technology in the many industries that drones are finding applications, including agriculture, conservation, deliveries, film, and public safety. Additionally, the development of robust gate detection algorithms could influence other applications for object detection where performance is critical, such as in autonomous vehicles, image search, and augmented reality experiences.