Vision-based hand signal recognition in construction: A feasibility study

Abstract

In construction fields, it is common for workers to rely on hand signals to communicate and express thoughts due to their simple but effective nature. However, the meaning of these hand signals was not always captured precisely. As a result, construction errors and even accidents were produced. This paper presented a feasibility study on investigating whether the hand signals could be captured and interpreted automatically with computer vision technologies. It starts with the literature review of existing hand gesture recognition methods for sign language understanding, human-computer interaction, etc. It is then followed by creating a dataset containing 11 classes of hand signals in construction. The performance of two state-of-the-art 3D convolutional neural networks is measured and compared. The results indicated that a high classification accuracy (93.3%) and a short inference time (0.17 s/gesture) could be achieved, illustrating the feasibility of using computer vision to automate hand signal recognition in construction.