Speaker adaptation for telephony data using speaker clustering

Abstract

This paper reports an ongoing effort to develop an unsupervised on-line speaker adaptation method for the telephony environment. All speakers in the training data corpus are acoustically pre-clustered into clusters, and a cluster-dependent system is built for each cluster. When a new telephony test speaker is given, a cluster, which is the closest to the speaker, is determined and selected by an improved distance measure. Based on this selected cluster, a MLLR (maximum likelihood linear regression) adaptation algorithm with block diagonal transformation is applied to move the cluster model to be closer to the testing speaker. For telephony applications the adaptation data can be very short or noisy, potentially the MLLR adapted means can be unreliable. A MAP-like weighting scheme for MLLR adaptation is applied to ensure the adapted mean reliability when the adaptation data is very short.