A Self-Supervised Learning Technique for Road Defects Detection Based on Monocular Three-Dimensional Reconstruction

Abstract

Abstract This paper presents a self-supervised learning technique for road surface defects detection using a monocular camera. The uniqueness of the proposed technique relies on its self-supervised learning structure which is achieved by combining physics-driven three-dimensional (3D) reconstruction with data-driven Convolutional Neural Network (CNN). Only images from one camera are needed as the inputs to the model without human labeling. The 3D point cloud are reconstructed from input images based on a near-planar road 3D reconstruction process to self-supervise the learning process. During testing, the network receives images and predicts the images as defect or non-defect. A refined class prediction is produced by combining the 3D road surface data with the network output when the belief of original network prediction is not strong enough to conclude the classification. Experiments are conducted on real road surface images to find the optimal parameters for this model. The testing results demonstrate the robustness and effectiveness of the proposed self-supervised road surface defects detection technique.