A Novel Approach for Improving Guitarists’ Performance Using Motion Capture and Note Frequency Recognition

Abstract

New guitarists face multiple problems when first starting out, and these mainly stem from a flood of information that they are presented with. Students also typically struggle with proper pitch frequency recognition and accurate left-hand motion. A variety of relevant solutions have been suggested in the existing literature; however, the majority have ultimately settled on two approaches. The first is finger motion capture, wherein researchers focus on extracting finger positions through analyzing images and videos. The second is note frequency recognition, wherein researchers focus on analyzing notes and frequencies from audio recordings. This paper proposes a novel hybrid solution that includes both finger motion capture and note frequency recognition in order to conduct a full assessment and give feedback on a guitarist’s performance. To classify hand positions, several classification algorithms are tested. The random forest algorithm obtained superior results, with an accuracy of 99% for overall hand movement and an average of 97.5% for the classification of each finger. Meanwhile, two algorithms were tested for note recognition, where the harmonic product spectrum (HPS) approach obtained the highest accuracy of 95%.