Assessing human worker performance by pattern mining of Kinect sensor skeleton data

Abstract

The human worker is an in-disposable factor in manufacturing processes. Traditional observation methods to assess their performance is time-consuming and expert-dependent, while it is still impossible to diagnose the detailed movement trajectory with the naked eye. Industry 4.0 technologies can innovate that process with smart sensors paired with data mining techniques for automated operation and develop a database of frequent movements for corporate reference and improvement. This paper proposes an approach to automatically assess worker performance with skeleton data by applying pattern mining methods and supervised learning algorithms. A use case is performed on an electrical assembly line to validate the approach, with the skeleton data collected by Kinect sensor v2. By using supervised learning, the movements of workers in each workstation can be segmented, and the line performance can be assessed. The work movement motifs can be recognized with pattern mining. The mined results can be used to further improve the production processes in terms of work procedures, movement symmetry, body utilization, and other ergonomics factors for both short and long-term human resource development. The promising result motivates further utilization of easy-to-adopt technology in Industry 5.0, which facilitates human-centric data-driven improvements.