Efficient approach to automated pavement manhole cover detection with modified faster R-CNN

Abstract

Abstract At the present time, increasing attention is being paid to the detection of road facilities, such as manhole covers— an important type of road facility which can have tangible impacts on driving safety and comfort. This paper proposes a robust method based on a modification of the Faster Region Convolutional Neural Network (Faster R-CNN) to automatically detect pavement manhole covers. We establish a manually annotated image library that consists of 1,245 manhole cover images collected by 1 mm laser imaging system, and implement the modified Faster R-CNN architecture to locate manhole covers exclusively under realistic and complex environments. Compared with the original Faster R-CNN, the proposed modification is to replace the feature extractor used in the original Faster R-CNN with a more-efficient backbone ResNet50, and implement a Feature Pyramid Network (FPN) to fuse multi-scale features. The experimental results demonstrate that the modified Faster R-CNN outperforms the original Faster R-CNN and other state-of-the-art models, including YOLOv4, EfficientDet, and YOLOX. The F1-score and Overall-IOU achieved by the modified Faster R-CNN on 250 test images are 98.15 per cent and 92.07 per cent respectively. To further verify the robustness of the proposed method, the modified Faster R-CNN is applied to process manhole cover images which are taken randomly by a smartphone and thus very different to the manhole cover images acquired by the laser imaging system. It is found that the modified Faster R-CNN can also yield similar detection efficiency even for images representing highly dissimilar viewing angles and unforeseen scenarios, implying the benefits of deep-learning-based object detection algorithms to intelligent investigation of pavement manhole covers.