Ground truth estimation of spoken english fluency score using decorrelation penalized low-rank matrix factorization

Abstract

In this paper, we propose ground truth estimation of spoken English fluency scores using decorrelation penalized low-rank matrix factorization. Automatic spoken English fluency scoring is a general classification problem. The model parameters are trained to map input fluency features to corresponding ground truth scores, and then used to predict a score for an input utterance. Therefore, in order to estimate the model parameters to predict scores reliably, correct ground truth scores must be provided as target outputs. However, it is not simple to determine correct ground truth scores from human raters' scores, as these include subjective biases. Therefore, ground truth scores are usually estimated from human raters' scores, and two of the most common methods are averaging and voting. Although these methods are used successfully, questions remain about whether the methods effectively estimate ground truth scores by considering human raters' subjective biases and performance metric. Therefore, to address these issues, we propose an approach based on low-rank matrix factorization penalized by decorrelation. The proposed method decomposes human raters' scores to biases and latent scores maximizing Pearson's correlation. The effectiveness of the proposed approach was evaluated using human ratings of the Korean-Spoken English Corpus.