Sewer pipe defect detection via deep learning with local and global feature fusion

Abstract

The damages triggered by the long-term corrosion, external disturbance, and uneven support pressure result in defective states of sewer pipes. Nowadays, the sewer pipe images are easily captured by closed-circuit televisions (CCTV) and quick-view (QV). However, the defect detection and grading still depend on human visual interpretation mainly, which are time-consuming, subjective and costing. Therefore, an efficient, accurate and automated method for sewer pipe defect localization and fine-grained classification is needed. To this end, this work introduces a novel two-stage learning-based method, to endow the capability of object detection network for the sewer pipe defect detection and fine-grained classification simultaneously, over all defect regions by exploiting the multi-layer global feature fusion techniques. Specifically, based on a two-stage object detection network, the strengthened region proposal network (SRPN) first generates representative region proposals by fusing multi-scale feature maps from the backbone network for defect region localization. Instead of only extracting the proposed regions for defect classification, we concatenate the proposed region feature and the global contextual feature from the corresponding image, to enhance feature representation of the fine-grained defect classification network. Extensive experiments demonstrate that our learning-based method gets the state-of-the-art performance for sewer pipe defect localization and fine-grained classification, compared with the several classical learning-based methods, according to the solid experiments. Furthermore, our method has been applied for sewer pipe inspection in many China cities.