Near real‐time bolt‐loosening detection using mask and region‐based convolutional neural network

Abstract

As an essential part of the steel structure, looseness of bolted joint greatly affects the integrity of the structure; however, there is a lack of technologies to detect the multibolt-loosening condition in practical applications. This study proposes an automatic method based on mask and region-based convolutional neural network to detect bolt looseness by labeling the defect to each pixel in an image, from which labeled classification types, locations, and geometric information can be obtained. In total, 300 original images and 700 images data augmentation were collected to prepare the dataset for training, validation, and testing. The dataset consists of three classes, that is, background, tight, and loose. With the help of a mask region-based convolution neural network (Mask RCNN), the testing results showed that the general precision and recall rates were 93.98% and 93.88%. To verify the accuracy and practicality of the trained detection model, the effect of photography angles, image shooting distances, and lighting conditions were experimentally tested. Results from a webcam and unmanned aerial vehicle (UAV)-assisted test have proven that the proposed method has potential to classify, segment, localize, and count the loosened bolts in near real-time for steel structures.