Real-Time Inspection System for Ballast Railway Fasteners Based on Point Cloud Deep Learning

Hao Cui,Qingzhou Mao,Qingwu Hu,Jian Li

doi:10.1109/access.2019.2961686

Hao Cui, Qingzhou Mao + Show 2 more

Open Access

https://doi.org/10.1109/access.2019.2961686

Copy DOI

Abstract

Rail fasteners are the most numerous components in railways and they should be inspected periodically. Manual inspection is currently a common solution, which is laborious and low-efficient. Some automatic inspection approaches are proposed. But for ballast railway fasteners inspection, debris, especially ballast along tracks may cover the fasteners, which is still a tricky problem. In this paper, a real-time inspection system for ballast railway fasteners based on point cloud deep learning is developed. Dense and precise point cloud of fastener is obtained from the structured light sensors in the system. The point cloud of fastener is segmented into different parts to avoid the interference of debris on fasteners. A ballast fastener point cloud semantic segmentation dataset is created based on automatic annotation method. Several deep learning point cloud segmentation models are tested in this dataset and PointNet++ is selected to be deployed in the real-time deep learning module of the system. Field tests on ballast railways show excellent accuracy and efficiency of this system.

Highlights

Railway is an important infrastructure for transportation, comparing with other transportation modes, railway transportation achieves a perfect balance in price, efficiency and capacity
Most methods are based on two-dimensional vision like images or video, Marino et al [9] built a real-time visual inspection system for hexagonal-head fasteners based on line-scan camera, this system can detect presence/absence of fasteners with an accuracy of 99.6% in detecting visible fasteners and of 95% in detecting missing fasteners
MODULES OF THE SYSTEM In this part, we provide details of the modules in the system, including structured light sensors, encoder, single board computer and deep learning module

Summary

INTRODUCTION

Railway is an important infrastructure for transportation, comparing with other transportation modes, railway transportation achieves a perfect balance in price, efficiency and capacity. Some researchers developed fastener inspection method based on three-dimensional (3D) point cloud from structured light sensors. In our previous work [18], [19], a structured-light-based fastener inspection system called Intelligent Rail Checker (IRC) was developed, this system could detect missing, defected and loose fasteners, and measure geometric parameters of fasteners. A fastener point cloud automatic annotation method based on region growing is proposed to create a ballast rail fastener point cloud semantic segmentation dataset. B. POINT CLOUD SEMANTIC SEGMENTATION DATASET OF BALLAST RAILWAY FASTENERS Training of deep learning neural networks for point cloud segmentation requires massive labeled data. Calculate the deviation between the main direction of spring bar (blue arrow in FIGURE 10) and the y axis of fastener coordinate, if the deviation angle is larger than 5◦, the spring bar will be considered skewed

EXPERIMENTS

Findings

CONCLUSION