A lip-tracking system based on morphological processing and block matching techniques

Abstract

This paper describes the application of image processing techniques in extracting the lip kinematics parameters (velocity and displacement) from image sequences. The centres of the lips are located by morphological image processing and cluster analysis. The motion of the lips is determined by a block matching algorithm. The paper presents a modified block matching algorithm which solves the problems caused by uniform shading and texture. The paper also describes a method which transforms the motion vectors into lip velocities and displacements. Moreover, the correlation between the lip information and the speech signals is demonstrated. Finally, the paper explains how the lip-tracking system can be applied to speech segmentation. The principal results show that lip information alone is not sufficient for speech segmentation. However, lip information may assist an audio speech segmentation system if the speech signals are corrupted by noise.