Road Pothole Detection Using Unmanned Aerial Vehicle Imagery and Deep Learning Technique

Abstract

Potholes are considered the main factor for road defects, which leads to road status deterioration, which, consequently will lead to increased road accidents. The first step in road maintenance is to inspect the road surface and then accurately detect potholes. However, manually identifying them is costly and time-consuming. In this study, unmanned aerial vehicle (UAV) imagery was used to create orthophotos of the roads and, using deep learning methods, potholes were detected. The used deep learning method in this study is the "you only look once" (YOLO) algorithm. YOLO is one of the "deep learning-based approaches" to detecting objects and is a single-stage network which requires only one forward propagation across the neural network and focuses on the entire image. The fourth version of YOLO is YOLOV4, which has two different architectures (YOLOv4 and YOLOv4-tiny). Two roads were chosen as the study areas, and to generate the orthophotos of the roads, UAV was used to acquire images. To train both methods in the process of detecting potholes using deep learning, 5300 images were used, 90% used for training and 10% applied for testing. The two used architectures were trained for 6000 iterations. Both methods were evaluated based on the average loss, mean average precision (mAP), and training and testing time. The results showed that the (mAP) values for YOLOv4 and YOLOv4-tiny were 91. 2% and 85.7%, respectively. At the end of the 6000 iterations, the average loss for YOLOv4 is 0.30% and for YOLOv4-tiny is 0.34%. In the training process, YOLOv4 needs 29 seconds for each iteration, while YOLOv4-tiny requires only 8 seconds. In the test process, YOLOv4-tiny is faster at detecting potholes than YOLOv4. The approaches were tested on orthophotos created by processing UAV photos. When comparing the detection of both architectures with visual detection, the results showed that YOLOv4 was able to detect most of the potholes on roads, but YOLOv4-tiny detected a lower number of potholes.