CUFuse: Camera and Ultrasound Data Fusion for Rail Defect Detection

Abstract

This paper proposes a multi-source data fusion algorithm for rail surface defect detection in both camera-based rail inspection images and ultrasound B-scan images. First, we design a rail surface segmentation algorithm based on image bilateral filtering, Sobel edge detection, and rail surface edge detection to extract the rail surface area. Second, we build a camera and ultrasound data fusion (CUFuse) model for rail surface defect detection, including two main networks: multi-source data feature extraction and multi-scale feature fusion networks. The multi-source data feature extraction network consists of two BoTNet 50 networks as feature extraction networks to extract five stages of features in camera-based images and ultrasound B-scan images. The multi-scale feature fusion network consists of five feature fusion modules to fuse the feature information output by the multi-source data feature extraction network. Finally, we use the CUFuse model to detect the rail surface defect dataset, and output five rail surface state types, including Light, Moderate, Severe, Normal, and Joint. The results show that the accuracy of the CUFuse model is 96.97%, which can accomplish the task of rail surface defect detection on railway sites.