Computer vision method for pianist's fingers information retrieval

Abstract

Voice, text or music recognition systems have quite a lot of elements in common with gesture recognition systems. They recognize something moving in orbit in space and in time. Hidden Markov Models (HMM) is an advisable mathematic theory managing stochastic processes such as human voice and gestures. This paper proposes a system retrieving music information from the right hand of a pianist using computer vision technology. Finger movement recognition is included in a general procedure, the gesture recognition. The results from a tentative analysis led to the use of a low cost video camera. The study on the pianist fingering retrieval has been effectuated on four aspects: preparation and capture of video signal, finger segmentation and tracking, feature vector exportation and HMM classifier. The recognition has not been effectuated in real time. Video signal processing (masks, filters, threshold etc) as well as the application of HMM was done with the use of suitable MATLAB toolboxes.