Real-Time Road Monitoring Using Deep Learning Algorithm Deployed on IoT Devices

Abstract

Poor road conditions play a significant role in road accidents as well as damage to the structural integrity of the vehicle. Detecting potholes on the roads is an essential step toward preventing road accidents. Pothole detection can be a challenging task due to their irregular shape and inconsistent size. With new innovations in artificial intelligence (AI), machine learning (ML), and deep learning (DL), it is possible to learn semantic features to address real-world problems in numerous applications. Internet of Thing (IoT) devices have proven to be a vital instrument for in situ measurements and source of real-time information. This chapter focuses a framework for automated detection and counting of potholes for real-time monitoring of roads using IoT devices integrated with a DL technique. The framework precisely detects and counts the potholes in real time, via video feed of the road from a vehicle-mounted dashcam connected to an IoT device. Detection of potholes is achieved using convolution neural network (CNN)-based algorithm, trained on more than 22,000 images of potholes. The study compares the efficacy of faster Region Based Convolutional Neural Networks (R-CNN) and single shot detection (SSD) MobileNet network architectures in the context of pothole detection and road monitoring under different illumination conditions. The models were compared based on their performance against accuracy and inference speed. In case of accuracy, Faster R-CNN outperformed the SSD MobileNet, with f-score averaging 93.59% under different illumination conditions, whereas SSD MobileNet depicted accuracy of 89.58%. However, SSD MobileNet could detect the potholes faster. The model is further coupled with DeepSORT tracking algorithm to track and count the potholes detected; the tracking algorithm also ensures that each pothole in the video frame is accounted only once. The model was deployed into Raspberry Pi single-board micro-computer for real-time road monitoring. In order to enhance the inference speed on the IoT device, Edge Tensor processing unit (TPU)-based machine learning acceleration is implemented using Google coral USB accelerator. The real-time inspection results show that the proposed “pothole detection” system achieves an accuracy of 89.58% at seven frames per second. The framework developed is the part of research on integration of IoT with machine learning; the framework provides scope for future researchers to enhance and carry out value addition of information on many other parameters for the benefit of research community and society.