Abstract
Accurate face registration is a key step for several image analysis applications. However, existing registration methods are prone to temporal drift errors or jitter among consecutive frames. In this paper, we propose an iterative rigid registration framework that estimates the misalignment with trained regressors. The input of the regressors is a robust motion representation that encodes the motion between a misaligned frame and the reference frame(s), and enables reliable performance under non-uniform illumination variations. Drift errors are reduced when the motion representation is computed from multiple reference frames. Furthermore, we use the L2 norm of the representation as a cue for performing coarse-to-fine registration efficiently. Importantly, the framework can identify registration failures and correct them. Experiments show that the proposed approach achieves significantly higher registration accuracy than the state-of-the-art techniques in challenging sequences.
Highlights
Face registration is the process of compensating for rigid transformations caused by head, body or camera movements in an image sequence. This is a fundamental pre-processing step for applications that interpret the non-rigid motions of facial features, such as facial action recognition [1], visual speech recognition [2], emotion recognition [3] and micro-expression recognition [4]
Significant drift errors may accumulate over time with online registration, even when individual registration errors remain under a tolerance threshold, leading to registration failures
In categories where there are multiple methods, we select experimentally the best-performing ones for the comparison: (i) the SURF-based method as the keypoint-based method, which generally outperformed the MSER method; (ii) the Robust fast Fourier transform (FFT) (R-FFT) method [24] as the transformationbased method, which, to the best of our knowledge, is the only method that proved robust against illumination variations and other outliers; (iii) the GradCorr method [7] as the direct method, which outperformed a number of Lucas-Kanade (LK) variants, namely IC-LK [6], ECC-LK [9] and Fourier-LK [25]
Summary
Face registration is the process of compensating for rigid transformations caused by head, body or camera movements in an image sequence. We argue that optimisation based on learning is promising for rigid facial registration, as invariance to non-rigid motions can be improved by training with sequences that contain facial activity. We propose to use optimisation via statistical learning for rigid facial registration. The proposed iterative framework (Fig. 1) reduces drift errors by computing Gabor motion energy with respect to multiple reference frames, and can identify and correct registration failures via probabilistic learning. Notable is the part-based registration performance in the presence of large facial activity due to facial expressions, and its robustness to non-uniform illumination variations.
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callMore From: IEEE transactions on image processing : a publication of the IEEE Signal Processing Society
Disclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
