An incrementally cascaded broad learning framework to facial landmark tracking

Abstract

Facial landmark tracking often adopts static models to generically fit per frame of video. This is considered inappropriate since such models ignore the informative correlation between previous and current frames. Moreover, most of these methods fail to balance the speed and accuracy for video-based facial landmark tracking simultaneously. In this paper, we propose an efficient online framework for video-based face alignment, named Incrementally Cascaded Broad Learning framework (ICBL). ICBL aims to continuously enhance the prediction capability of tracking model for sequential data. It is capable of learning the spatial appearance on specific-person statistics from continuous facial frames and using such knowledge to incrementally tune a cascade of regressors in parallel. To achieve this goal, we approximate the facial shape space by sampling from a dynamic distribution which is continuously updated by person-specific statistics from the tracked facial frames. This dramatically facilitates cascade regression to incrementally update all cascade-regressors in parallel, thus allowing a fast update of the whole model. Furthermore, we successfully incorporate both the linear and non-linear mappings into our parallel cascade framework and introduce Broad Learning (BL) algorithm as a solution for them simultaneously. Experimental results on the most popular and large-scale benchmark for facial landmark tracking show highly competitive performance of proposed ICBL in comparisons with the state-of-the-arts. The code of our ICBL framework has been available fromhttps://github.com/CaifengLiu/Facial-landmark-tracking-by-ICBL.