Click-event sound detection in automotive industry using machine/deep learning

Abstract

In the automotive industry, despite the robotic systems on the production lines, factories continue employing workers in several custom tasks getting for semi-automatic assembly operations. Specifically, the assembly of electrical harnesses of engines comprises a set of connections between electrical components. Despite the task is easy to perform, employees tend not to notice that a few components are not being connected properly due to physical fatigue provoked by repetitive tasks. This yields a low quality of the assembly production line and possible hazards. In this work, we propose a sound detection system based on machine/deep learning (ML/DL) approaches to identify click sounds produced when electrical harnesses are connected. The purpose of this system is to count the number of connections properly made and to feedback to the employees. We collect and release a public dataset of 25,000 click sounds of 25 ms length at 22 kHz during three months of assembly operations in an automotive production line located in Mexico. Then, we design an ML/DL-based methodology for click sound detection of assembled harnesses under real conditions of a noisy environment (noise level ranging from −16.67 dB to −12.87 dB) including other machinery sounds. Our best ML/DL model (i.e., a combination between five acoustic features and an optimized convolutional neural network) is able to detect click sounds in a real assembly production line with an accuracy of 94.55±0.83 %. To the best of our knowledge, this is the first time a click sounds detection system in assembling electrical harnesses of engines for giving feedback to the workers is proposed and implemented in a real-world automotive production line. We consider this work valuable for the automotive industry on how to apply ML/DL approaches for improving the quality of semi-automatic assembly operations.