Virtual guitar: using real-time finger tracking for musical instruments

Abstract

Kinect, a 3D sensing device from Microsoft, invokes the human-computer interaction research evolution. Kinect was implemented in many areas including music. One was implemented in a virtual musical instrument (VMI) system, which uses natural gestures to produce synthetic sounds similar to real musical instruments. From related work, it was found that the use of a large joint such as hand, arm and leg is inconvenient and limits the way of playing VMI. Thus, this study proposes a fast and reliable finger tracking algorithm suitable for virtual musical instrument playing. In addition, a virtual guitar system application is developed as an implementation of the proposed algorithm. Experimental results show that the proposed method can be used to play many varieties of songs with an acceptable quality. Furthermore proposed application could be used by a beginner who does not have any experience in music or on playing a real musical instrument.