Automatic Lip Tracking and Action Units Classification using Two-Step Active Contours and Probabilistic Neural Networks

Abstract

The most of the human emotions are communicated by changes in one or two of discrete facial features. Theses changes are coded as action units (AUs). Among the facial features mouth has most flexible deformability and it is highly complicated to track. In this paper, we develop a lip shape extraction and lip motion tracking system both in static and dynamic facial images, based on a novel two step active contours model. A knowledge based system is used for estimating initial position of mouth. An oval shaped initial active is considered inside the estimated mouth region. At the first step active contour locks onto stronger upper lip edges by using both high threshold Canny edge detector and balloon energy for contour deflation. Then using lower threshold image gradient as well as balloon energy for inflation, snake inflates and locks onto weaker lower lip edges. Extracted lip feature points are used to extract some geometric features to form a feature vector which is used to classify lip images into AUs, using probabilistic neural networks (PNN). Experimental results show robust edge detection and reasonable classification where an average AUs recognition rate is 85.98% in image sequences and 77.44% in static images