Smart monitoring and automated real-time visual inspection of a sealant applications (SMART-VIStA)

Abstract

Computer Vision (CV) assisted robotics applications are receiving significant attention with increased automation on the factory floor. The automated solutions provide a pathway to smart manufacturing and improve the overall product quality inspection process. We propose a closed-loop framework – Smart Monitoring and Automated Real-Time Visual Inspection of Sealant Application (SMART-VIStA) to address critical challenges in process automation and optimized process parameters feedback. This novel modular approach combines CV-based robotic path planning, deep learning-based classification, image segmentation, and real-time recommendations for corrective actions in a sealant deposition process. Specifically, the system includes pose detection and localization for the rectangular deposition plate, predicting and segmenting the glue dot class through few-shot learning, quantifying the quality of the glue dot through the unique shape quality index for circular artifacts, and an advisory recommendation system through Bayesian Decision Network. Monitoring of real-time results through a graphical user interface (GUI) allows the flexibility to change the parameters for subsequent cycles. A prototype for this process has been set up at the Boeing manufacturing facility to demonstrate its effectiveness. With accuracies extending beyond 90% in multiple tasks, this framework has promising applications in sealant inspection and can extend to different manufacturing scenarios, such as robotic welding and robotic painting.