Real-Time Object Detection and Recognition Using Fixed-Wing LALE VTOL UAV

Abstract

This research aimed to identify objects from a real-time video (30 Hz) stream transmitted from a low-altitude long-endurance (LALE) fixed-wing hybrid vertical takeoff and landing (VTOL) unmanned aerial vehicle (UAV) designed for asset monitoring. Illumination, rotation, and scale variations complicate the video frames created by UAVs’ optical payload, requiring preprocessing. Detection, identification, and classification of different objects (or targets) from a real-time video feed require fast and reliable detection algorithms, models, and computation. The weight restrictions and the onboard power supply only allowed for a Raspberry Pi onboard the UAV, thereby seriously limiting computational power. Image frames were created from the video data after enhancing its quality using image processing algorithms. Then, anchor-based object detection algorithms were applied to the processed and enhanced image frames. The Raspberry Pi at airspeeds of 16–18 m/s requires fast object detection algorithms having minimal computational overhead. A supervised learning technique using a supporting library further augmented the real-time detection at the ground control station (GCS). Four scenarios trained the candidate algorithms, namely: 1) helipad; 2) green area; 3) burned area; and 4) under construction area. The single-shot multibox detector (SSD) aided the supervised learning process during trials utilizing the labeled image datasets. The comparison of detection and identification performance of the SSD with ResNet-50 and ResNet-101 backbones with other methods, such as Faster-RCNN with four backbone architectures, namely: 1) ResNet-101; 2) ResNet-50; 3) deformable ResNet; and 4) FPN in four detection scenarios indicated better precision and reduced detection delays.