Mobile crane signalman static hand signals classification framework using deep convolution neural network

Abstract

Cranes are the need of every construction site as the construction paradigm is shifting from traditional (on-site) methods toward an off-site (modularization) approach. The communication between the crane operator and the crane signalman plays a significant role to complete the construction project safely and efficiently. The communication between crane operator and signalman relies on hand signals and two-way radio communication systems. However, these means of communication are not reliable in modern construction as the construction sites are more congested and noisy. Any miscommunication may lead to a disastrous accident on the construction site. The recent advancement in information technology can assist to add more layers of communication in the crane Industry. This paper presents a framework that uses deep convolutional neural network (DCNN) architecture for static hand signal classification using the crane signalman hand signals dataset. The DCNN model was developed to classify 18 different crane signalman hand signals. The model was trained, validated, and tested using a dataset of 8133 images, and achieved average accuracies of 89.1% and 84.6% for the training dataset and the validation dataset, respectively. The precision, recall, and F1 score in the test dataset were recorded as 81.5%, 81.8%, and 81.7%, respectively. The model is further validated with real-time hand signals classification and an accuracy of 87.9% is achieved. This developed framework can be used as another layer of communication with the current state of practice to reduce the communication error between crane signalman and operator.