Tiny machine learning empowers climbing inspection robots for real-time multiobject bolt-defect detection

Abstract

Ensuring the structural integrity of steel construction is critical, necessitating effective methods for detecting bolt defects. Traditional inspection methods are reliable but require significant labor and struggle with the accessibility of complex structures. This research introduces a real-time bolt-defect detection system that integrates tiny machine learning (TinyML) with a magnetic climbing robot to enhance inspection efficiency and scope. The system employs the faster objects, more objects (FOMO) algorithm optimized for edge computing on microcontrollers. This approach enables accurate bolt identification, including normal, loose, and missing bolts. The average system accuracy is 82%, with precision and recall values from 0.57 to 0.89 and 0.67 to 0.87, respectively. The system demonstrates balanced detection, evidenced by an F1 score improvement of up to 62%. The FOMO model displays compelling performance on defect detection tasks, achieving an F1 score of approximately 75%, outperforming the MobileNetV2 Single Shot Multibox Detector with Feature Pyramid Networks Lite and You Only Look Once version 5 small. The efficiency of the FOMO model is highlighted by its low hardware requirements at less than 0.1 MB of flash memory and 893.8 KB of random access memory for the 32-bit floating-point data format and is reduced for the 8-bit integer data format, with inference times of 142 ms and 86 ms, respectively. These findings contrast with the higher resource demands and slower inference time of the compared models, indicating the suitability of the FOMO model for low-capacity microcontrollers and its feasibility for real-time applications. The performance analysis across scenarios confirms the high precision (average 0.77) and recall (average 0.76), validating the robustness of the model in diverse environmental conditions. The system offers an advancement over traditional bolt-defect detection methods in accuracy and efficiency by leveraging TinyML and a magnetic climbing robot, setting new benchmarks for real-time inspection technology. The quantitative results underscore the performance of the system, presenting a scalable, cost-effective solution for improving the safety and durability of steel structures.