Kernel-based Persian viseme clustering

Abstract

Viseme (Visual Phoneme) clustering and analysis in every language is among the most important preliminaries for conducting various multimedia researches as talking head, lip reading, lip synchronization and computer assisted pronunciation training applications. With respect to the fact that clustering and analyzing visemes are language dependent processes, we concentrated our research on Persian language, which indeed has suffered from lack of such study. In this paper, we used a hierarchical approach for clustering visemes in Persian language based on principal component analysis of a polynomial kernel matrix considering coarticulation effect. Having obtained feature vector of each phoneme, we applied unweighted pair group method with arithmetic mean to each projected viseme on constructed manifold. Then furthest neighbor of the weight value as a result of reconstruction is set as the criterion for comparing viseme dissimilarity. In order to indicate the robustness of the proposed algorithm, a set of experiments was conducted on Persian databases in which two syllables were examined. Comparing the results of the clustering algorithm with that of the perceptual test given by an expert proves a reasonable evaluation of the proposed algorithm.