Automatic lip localization under face illumination with shadow consideration

Abstract

Lip-reading has potential attractive applications in information security, speech recognition, secret communication and so forth. To build an automatic lip-reading system, one key issue is how to locate the lip region, particularly under the changing illumination condition. Empirical studies have shown that the recognition rate of a lip-reading system greatly relies on the accuracy of the lip localization. Unfortunately, to the best of our knowledge, lip localization under face illumination with shadow consideration has not been well solved yet. Moreover, this problem is also one of the major obstacles to keeping an automatic lip-reading system from the practical applications. This paper therefore concentrates on this problem and proposes a new approach to obtain the minimum enclosing rectangle surround of a mouth automatically based upon the transformed gray-level image. In this approach, a pre-processing is firstly made to reduce the interference caused by shadow and enhance the boundary region of lip, through which the left and right mouth corners are estimated. Then, by building a binary sequence based on the gray-level values along with the vertical midline of mouth, the top and bottom crucial points can be estimated. Experiments show the promising result of the proposed approach in comparison with the existing methods.