Lip segmentation and tracking for facial palsy

Abstract

We developed the asymmetry analyzing system for facial palsy patient's rehabilitation progress study. Using PC standard imaging device, captured 640*480 RGB image is converted into HSV space. A Lip-shape mask is extracted by thresholding. By taking 5 regions consisted in one region on lip and four regions on face skin, reasonable thresholds are determined by Fuzzy C-Means clustering. The extreme points on the lip shape mask are extracted to get the seeds for tracking. Segmented seed points are tracking by Iterative Lucas-Kanade tracking method in pyramids at 30 fps and recording simultaneously. To reduce the disk writing load on computer, we use asynchronous mode file writing, which is going to transfer to and review by clinician. Tracking shows quite reliable results, but sometimes the tracked points are following along the lip line because of the similar contrasts. Therefore, the first strategy to improve the reliability of tracking is using the high contrast points, such as left and right maximal point of lip shape. The second is clustering some points near the maximal points and eliminating outlying tracking points. The third is rechecking the lip shape using lip segmentation when the operator confirms that subject's maximal lip moving. Left and right tracking points are compared in forms of trajectory plot.