An automatic singing skill evaluation method for unknown melodies using pitch interval accuracy and vibrato features

Abstract

This paper presents a method of evaluating singing skills that does not require score information of the sung melody. This requires an approach that is different from existing systems, such as those currently used for Karaoke systems. Previous research on singing evaluation has focused on analyzing the characteristics of singing voice, but were not aimed at developing an automatic evaluation method. The approach presented in this study uses pitch inter- val accuracy and vibrato as acoustic features which are indepen- dent from specific characteristics of the singer or melody. The ap- proach was tested by a 2-class (good/poor) classification test with 600 song sequences, and achieved an average classification rate of 83.5%. The aim of this study is to explore a method of automatic evalua- tion of singing skills without score information. Our interest lies in identifying the criteria that human subjects use in judging the quality of singing for unknown melodies, using acoustic features which are independent from specific characteristics of the singer or melody. Such evaluation systems can be useful tools for im- proving singing skills, and also can be applied to broadening the scope of music information retrieval and singing voice synthesis. Previous work related to singing skills include those based on a control model of fundamental frequency (F0) trajectory (1), gen- eral characteristics (2, 3), as well as work on automatic discrimi- nation of singing and speaking voices (4), and acoustic differences between trained and untrained singers' voices (5, 6, 7). None of these work have gone as far as presenting an automatic evaluation method. This paper presents a singing skill evaluation scheme based on pitch interval accuracy and vibrato, which are regarded as features that function independently from the individual characteristics of singer or melody. To test the validity of these features, an experi- ment of automatic evaluation of singing performance by a 2-class classification (good/poor) was conducted. The following sections describe our approach and the exper- imental results of its evaluation. Section 2 presents discussion of features. Section 3 describes the classification experiment and its evaluation. Section 4 concludes the paper, with discussion on di- rections for future work.