A Two-Stage Focal Transformer for Human–Robot Collaboration-Based Surface Defect Inspection

Abstract

Abstract Human–robot collaboration has become a hotspot in smart manufacturing, and it also has shown the potential for surface defect inspection. The robot can release workload, while human collaboration can help to recheck the uncertain defects. However, the human–robot collaboration-based defect inspection can be hardly realized unless some bottlenecks have been solved, and one of them is that the current methods cannot decide which samples to be rechecked, and the workers can only recheck all of the samples to improve inspection results. To overcome this problem and realize the human–robot collaboration-based surface defect inspection, a two-stage Transformer model with focal loss is proposed. The proposed method divides the traditional inspection process into detection and recognition, designs a collaboration rule to allow workers to collaborate and recheck the defects, and introduces the focal loss into the model to improve the recognition results. With these improvements, the proposed method can collaborate with workers by rechecking the defects and improve surface quality. The experimental results on the public dataset have shown the effectiveness of the proposed method, the accuracies are significantly improved by the human collaboration, which are 1.70%∼4.18%. Moreover, the proposed method has been implemented into a human–robot collaboration-based prototype to inspect the carton surface defects, and the results also verify the effectiveness. Meanwhile, the proposed method has a good ability for visualization to find the defect area, and it is also conducive to defect analysis and rechecking.