Collaborative Learning Attention Network Based on RGB Image and Depth Image for Surface Defect Inspection of No-Service Rail

Abstract

Surface defect inspection of no-service rail is important for safety of railway transportation. However, there are several challenges of irregular defect boundary, similar foreground and background for no-service rail surface defect inspection. To deal with the above challenges, depth image is used to provide complementary spatial information to RGB image. In recent years, with the development of deep learning and computer vision technology, intelligent inspection of defect has made great progress. We propose a neural network named collaborative learning attention network (CLANet) for no-service rail surface defect inspection. Our method can inspect the defect object of rail surface and segment the accurate region of that defect. The proposed method consists of three main stages: feature extraction, cross-modal information fusion, and defect location and segmentation. A multimodal attention block is proposed to highlight complex defect object with a new cross-modal fusion strategy. Furthermore, dual stream decoder enriches the representation of advanced features and avoids the dilution of information in the decoding stage. Suffering from the scarcity of defective data, an industrial RGB-D dataset NEU RSDDS-AUG is built. Finally, ablation studies verify the effectiveness of our proposed method. Compared with the existing nine state-of-the-art methods, CLANet has achieved improvements in all five parameters. Our method is also competitive on four public benchmark datasets.