Towards robotic disassembly: A comparison of coarse-to-fine and multimodal fusion screw detection methods

Abstract

To accelerate the transition towards a more circular economy it is believed essential to increase the economic viability and hence the adoption of de- and remanufacturing processes. For this, industry 4.0 technologies, including robotization, computer vision and artificial intelligence, are commonly considered as the main enablers to reduce costs while boosting performances of de- and remanufacturing processes. However, the essential step in de- and remanufacturing of screw detection for robotic disassembly is still confronted with various challenges, such as false negatives due to deep screw holes and false positives caused by similar patterns and colors, e.g. letters in labels. Previous studies improved the detection accuracy by capturing images of only a fine part of the product and neglected the negative impact of this imaging approach on the overall time efficiency of the disassembly process. Therefore, two of the most distinct approaches are evaluated in the presented research: 1. A Coarse-to-Fine detection method, where potential screws across the entire product are detected by an area camera in the coarse stage to generate an optimal unscrewing path and screw types and locations are further refined in a second instance, the fine stage. 2. A multimodal fusion detection method, where a combination of a high-resolution color line and 3D triangulation line camera system captures full-product images allowing a single run screw detection in the RGB, Intensity and Depth (RGBID) images. Furthermore, a pooling-based aligned fusion module (PAlign) is introduced to improve the feature representation and mitigate the encountered spatial misalignment of different modality branches. Two industrially representative datasets comprising images of laptops and hard disk drives are collected for the training and evaluation of the screw detection and unscrewing task. Experimental results show that the RGBID fusion method with PAlign module achieves an unscrewing success rate of 80.4 % with an average detection and unscrewing speed of 2.27 s per screw, which is almost double the speed when compared to the Coarse-to-Fine approach. Through a detailed analysis comparing these most distinct screw detection methods in terms of imaging approach, lighting and camera hardware (1/2, area/line, low/high resolution, RGB/RGBID, 600/25.000€), this study provides concrete insights and recommendations aimed at enhancing the efficiency of disassembly systems for sustainable demanufacturing.