Stretchable e-Skin Patch for Gesture Recognition on the Back of the Hand

Abstract

Gesture recognition is important for human–computer interaction and a variety of emerging research and commercial areas including virtual and augmented reality. Current approaches typically require sensors to be placed on the forearm, wrist, or directly across finger joints; however, they can be cumbersome or hinder human movement and sensation. In this paper, we introduce a novel approach to recognize hand gestures by estimating skin strain with multiple soft sensors optimally placed across the back of the hand. A pilot study was first conducted by covering the back of the hand with 40 small 2.5 mm reflective markers and using a high-precision camera system to measure skin strain patterns for individual finger movements. Optimal strain locations are then determined and used for sensor placement in a stretchable e-skin patch prototype. Experimental testing is performed to evaluate the stretchable e-skin patch performance in classifying individual finger gestures and American Sign Language 0–9 number gestures. Results showed classification accuracies of 95.3% and 94.4% for finger gestures and American Sign Language 0–9 gestures, respectively. These results demonstrate the feasibility of a stretchable e-skin patch on the back of the hand for hand gesture recognition and their potential to significantly enhance human–computer interaction.