Facial feature detection with 3D convex local models

Abstract

This paper describes an improved system for locating facial features in images using constrained local models (CLM). CLM links a set of local patch classifiers via a PCA shape model for non-rigid alignment and tracking. The convex quadratic fitting (CQF) approach to CLM approximates the patch responses with quadratic functions, allowing the parameter updates to be calculated directly. The Bayesian CLM (BCLM) further extended this approach framing it as a Bayesian inference problem. We further extend the BCLM approach to enable the use of 3D shape models. A 3D shape model is preferred on theoretical grounds and improved performance is confirmed via an empirical evaluation. The extension to 3D is developed by first introducing a full similarity transform to the (linearized) 2D CQF error function. The minimization of this error function gives a set of parameter updates that can be combined with the current estimates via a compositional approach. The adaptation of the algorithm to 3D then follows directly. The resulting algorithm is evaluated on the labeled faces in the wild (LFW) dataset and the results show improved performance over both 2D BCLM and 3D CLM.