Shape Regression Research Articles

Parametric Regression on the Grassmannian.

We address the problem of fitting parametric curves on the Grassmann manifold for the purpose of intrinsic parametric regression. We start from the energy minimization formulation of linear least-squares in Euclidean space and generalize this concept to general nonflat Riemannian manifolds, following an optimal-control point of view. We then specialize this idea to the Grassmann manifold and demonstrate that it yields a simple, extensible and easy-to-implement solution to the parametric regression problem. In fact, it allows us to extend the basic geodesic model to (1) a "time-warped" variant and (2) cubic splines. We demonstrate the utility of the proposed solution on different vision problems, such as shape regression as a function of age, traffic-speed estimation and crowd-counting from surveillance video clips. Most notably, these problems can be conveniently solved within the same framework without any specifically-tailored steps along the processing pipeline.

M3 CSR: Multi-view, multi-scale and multi-component cascade shape regression

Automatic face alignment is a fundamental step in facial image analysis. However, this problem continues to be challenging due to the large variability of expression, illumination, occlusion, pose, and detection drift in the real-world face images. In this paper, we present a multi-view, multi-scale and multi-component cascade shape regression (M3CSR) model for robust face alignment. Firstly, face view is estimated according to the deformable facial parts for learning view specified CSR, which can decrease the shape variance, alleviate the drift of face detection and accelerate shape convergence. Secondly, multi-scale HoG features are used as the shape-index features to incorporate local structure information implicitly, and a multi-scale optimization strategy is adopted to avoid trapping in local optimum. Finally, a component-based shape refinement process is developed to further improve the performance of face alignment. Extensive experiments on the IBUG dataset and the 300-W challenge dataset demonstrate the superiority of the proposed method over the state-of-the-art methods.

Predicting infant cortical surface development using a 4D varifold-based learning framework and local topography-based shape morphing.

Longitudinal neuroimaging analysis methods have remarkably advanced our understanding of early postnatal brain development. However, learning predictive models to trace forth the evolution trajectories of both normal and abnormal cortical shapes remains broadly absent. To fill this critical gap, we pioneered the first prediction model for longitudinal developing cortical surfaces in infants using a spatiotemporal current-based learning framework solely from the baseline cortical surface. In this paper, we detail this prediction model and even further improve its performance by introducing two key variants. First, we use the varifold metric to overcome the limitations of the current metric for surface registration that was used in our preliminary study. We also extend the conventional varifold-based surface registration model for pairwise registration to a spatiotemporal surface regression model. Second, we propose a morphing process of the baseline surface using its topographic attributes such as normal direction and principal curvature sign. Specifically, our method learns from longitudinal data both the geometric (vertices positions) and dynamic (temporal evolution trajectories) features of the infant cortical surface, comprising a training stage and a prediction stage. In the training stage, we use the proposed varifold-based shape regression model to estimate geodesic cortical shape evolution trajectories for each training subject. We then build an empirical mean spatiotemporal surface atlas. In the prediction stage, given an infant, we select the best learnt features from training subjects to simultaneously predict the cortical surface shapes at all later timepoints, based on similarity metrics between this baseline surface and the learnt baseline population average surface atlas. We used a leave-one-out cross validation method to predict the inner cortical surface shape at 3, 6, 9 and 12 months of age from the baseline cortical surface shape at birth. Our method attained a higher prediction accuracy and better captured the spatiotemporal dynamic change of the highly folded cortical surface than the previous proposed prediction method.

Robust Eye Center Localization through Face Alignment and Invariant Isocentric Patterns.

The localization of eye centers is a very useful cue for numerous applications like face recognition, facial expression recognition, and the early screening of neurological pathologies. Several methods relying on available light for accurate eye-center localization have been exploited. However, despite the considerable improvements that eye-center localization systems have undergone in recent years, only few of these developments deal with the challenges posed by the profile (non-frontal face). In this paper, we first use the explicit shape regression method to obtain the rough location of the eye centers. Because this method extracts global information from the human face, it is robust against any changes in the eye region. We exploit this robustness and utilize it as a constraint. To locate the eye centers accurately, we employ isophote curvature features, the accuracy of which has been demonstrated in a previous study. By applying these features, we obtain a series of eye-center locations which are candidates for the actual position of the eye-center. Among these locations, the estimated locations which minimize the reconstruction error between the two methods mentioned above are taken as the closest approximation for the eye centers locations. Therefore, we combine explicit shape regression and isophote curvature feature analysis to achieve robustness and accuracy, respectively. In practical experiments, we use BioID and FERET datasets to test our approach to obtaining an accurate eye-center location while retaining robustness against changes in scale and pose. In addition, we apply our method to non-frontal faces to test its robustness and accuracy, which are essential in gaze estimation but have seldom been mentioned in previous works. Through extensive experimentation, we show that the proposed method can achieve a significant improvement in accuracy and robustness over state-of-the-art techniques, with our method ranking second in terms of accuracy. According to our implementation on a PC with a Xeon 2.5Ghz CPU, the frame rate of the eye tracking process can achieve 38 Hz.

Facial Feature Extraction with Its Applications

Purpose In the many face-related application such as head pose estimation, 3D face modeling, facial appearance manipulation, the robust and fast facial feature extraction is necessary. We present the facial feature extraction method based on shape regression and feature selection for real-time facial feature extraction. Materials and Methods The facial features are initialized by statistical shape model and then the shape of facial features are deformed iteratively according to the texture pattern which is selected on the feature pool. Results We obtain fast and robust facial feature extraction result with error less than 4% and processing time less than 12 ms. The alignment error is measured by average of ratio of pixel difference to inter-ocular distance. Conclusion The accuracy and processing time of the method is enough to apply facial feature based application and can be used on the face beautification or 3D face modeling.

The use of novel phenotyping methods for validation of equine conformation scoring results

In this experiment, which is based on a cohort of 44 Lipizzan mares from the Austrian state stud farm of Piber, we present new statistical techniques for the analysis of shape and equine conformation using image data. In addition, we examined which strategies and procedures of image processing techniques led to a successful interpretation of the traits implemented in horse breeding programs. A total of 246 two-dimensional anatomical and somatometric landmarks were digitized from standardized photographs, and the variation of shape has been analyzed by the use of generalized orthogonal least-squares Procrustes (generalized Procrustes analysis (GPA)) procedures. The resulting shape variables have been regressed on the results from linear type trait classifications. In addition, the rating scores of six conformation classifiers were tested for agreement, yielding an inter-rater correlation (inter-class correlation) ranging from 0.41 to 0.68, respectively, a κ coefficient ranging from 0.16 to 0.53. From the 12 linear type traits assessed on a valuating scale, only the type-related traits (type, breed-type and harmony) revealed significant (P<0.05) results in the regression analysis of shape variables on linear type traits. The other nine traits were characterized by a lower agreement between classifiers and did not result in a significant ‘shape regression’. Finally, the ‘horse shape space’ defined by shape variables resulting from GPA procedures offered the possibility to assist in trait definition and in the evaluation of ratings, and it is an adequate biological and objective scale to human perception of conformation, which is expressed in numerical data only.

Prediction of Longitudinal Development of Infant Cortical Surface Shape Using a 4D Current-Based Learning Framework.

Understanding the early dynamics of the highly folded human cerebral cortex is still an actively evolving research field teeming with unanswered questions. Longitudinal neuroimaging analysis and modeling have become the new trend to advance research in this field. However, this is challenged by a limited number of acquisition timepoints and the absence of inter-subject matching between timepoints. In this paper, we propose a novel framework that unprecedentedly solves the problem of predicting the dynamic evolution of infant cortical surface shape solely from a single baseline shape based on a spatiotemporal (4D) current-based learning approach. Specifically, our method learns from longitudinal data both the geometric (vertices positions) and dynamic (temporal evolution trajectories) features of the infant cortical surface, comprising a training stage and a prediction stage. In the training stage, we first use the current-based shape regression model to set up the inter-subject cortical surface correspondences at baseline of all training subjects. We then estimate for each training subject the diffeomorphic temporal evolution trajectories of the cortical surface shape and build an empirical mean spatiotemporal surface atlas. In the prediction stage, given an infant, we first warp all training subjects onto its baseline cortical surface. Second, we select the most appropriate learnt features from training subjects to simultaneously predict the cortical surface shapes at all later timepoints from its baseline cortical surface, based on closeness metrics between this baseline surface and the learnt baseline population average surface atlas. We used the proposed framework to predict the inner cortical surface shape at 3, 6 and 9 months from the cortical shape at birth in 9 healthy infants. Our method predicted with good accuracy the spatiotemporal dynamic change of the highly folded cortex.

顔特徴点検出における形状回帰モデルの適応的設計

顔特徴点検出における形状回帰モデルの適応的設計

3-D Facial Landmark Localization With Asymmetry Patterns and Shape Regression from Incomplete Local Features.

We present a method for the automatic localization of facial landmarks that integrates nonrigid deformation with the ability to handle missing points. The algorithm generates sets of candidate locations from feature detectors and performs combinatorial search constrained by a flexible shape model. A key assumption of our approach is that for some landmarks there might not be an accurate candidate in the input set. This is tackled by detecting partial subsets of landmarks and inferring those that are missing, so that the probability of the flexible model is maximized. The ability of the model to work with incomplete information makes it possible to limit the number of candidates that need to be retained, drastically reducing the number of combinations to be tested with respect to the alternative of trying to always detect the complete set of landmarks. We demonstrate the accuracy of the proposed method in the face recognition grand challenge database, where we obtain average errors of approximately 3.5 mm when targeting 14 prominent facial landmarks. For the majority of these our method produces the most accurate results reported to date in this database. Handling of occlusions and surfaces with missing parts is demonstrated with tests on the Bosphorus database, where we achieve an overall error of 4.81 and 4.25 mm for data with and without occlusions, respectively. To investigate potential limits in the accuracy that could be reached, we also report experiments on a database of 144 facial scans acquired in the context of clinical research, with manual annotations performed by experts, where we obtain an overall error of 2.3 mm, with averages per landmark below 3.4 mm for all 14 targeted points and within 2 mm for half of them. The coordinates of automatically located landmarks are made available on-line.

Hierarchical facial landmark localization via cascaded random binary patterns

The main challenge of facial landmark localization in real-world application is that the large changes of head pose and facial expressions cause substantial image appearance variations. To avoid high dimensional facial shape regression, we propose a hierarchical pose regression approach, estimating the head rotation, face components, and facial landmarks hierarchically. The regression process works in a unified cascaded fern framework with binary patterns. We present generalized gradient boosted ferns (GBFs) for the regression framework, which give better performance than ferns. The framework also achieves real time performance. We verify our method on the latest benchmark datasets and show that it achieves the state-of-the-art performance.

Prostate boundary segment extraction using cascaded shape regression and optimal surface detection.

In this paper, we proposed a new method (CSR+OSD) for the extraction of irregular open prostate boundaries in noisy extracorporeal ultrasound image. First, cascaded shape regression (CSR) is used to locate the position of prostate boundary in the images. In CSR, a sequence of random fern predictors are trained in a boosted regression manner, using shape-indexed features to achieve invariance against position variations of prostate boundaries. Afterwards, we adopt optimal surface detection (OSD) to refine the prostate boundary segments across 3D sections globally and efficiently. The proposed method is tested on 162 ECUS images acquired from 8 patients with benign prostate hyperplasia. The method yields a Root Mean Square Distance of 2.11±1.72 mm and a Mean Absolute Distance of 1.61±1.26 mm, which are lower than those of JFilament, an open active contour algorithm and Chan-Vese region based level set model, respectively.

GEODESIC REGRESSION OF IMAGE AND SHAPE DATA FOR IMPROVED MODELING OF 4D TRAJECTORIES.

A variety of regression schemes have been proposed on images or shapes, although available methods do not handle them jointly. In this paper, we present a framework for joint image and shape regression which incorporates images as well as anatomical shape information in a consistent manner. Evolution is described by a generative model that is the analog of linear regression, which is fully characterized by baseline images and shapes (intercept) and initial momenta vectors (slope). Further, our framework adopts a control point parameterization of deformations, where the dimensionality of the deformation is determined by the complexity of anatomical changes in time rather than the sampling of the image and/or the geometric data. We derive a gradient descent algorithm which simultaneously estimates baseline images and shapes, location of control points, and momenta. Experiments on real medical data demonstrate that our framework effectively combines image and shape information, resulting in improved modeling of 4D (3D space + time) trajectories.

A Failure-Aware Explicit Shape Regression Model for Facial Landmark Detection in Video

Facial landmark detection is fundamental for various face-related applications such as interactive avatars on mobile devices. Awareness of detection failure is critical for practical applications because even occasional failures to detect facial landmarks lead to bad user experience. This letter proposes a fast and robust AdaBoost Based Cascade Detector (ABCD) for discerning failures from shape regression in video on mobile devices. A vector of randomly sampled pixel intensities near facial landmarks is taken as the input feature for AdaBoost classifiers. Several AdaBoost classifiers are cascaded together for robustness, computational efficiency and to augment the theoretical number of false samples in training. With this failure detector, the correctly estimated shape of the previous frame can be utilized in the next frame for initialization, which not only improves the regression accuracy but also saves on face searching time. ABCD is incorporated into a recently proposed facial landmark detection algorithm Face Alignment by Explicit Shape Regression (FAESR). Experiments on videos show that failure awareness powered FAESR yields an accurate and automatic facial landmark detection with very low computational costs, which is suitable for real time application on mobile devices.

Modeling the effect of variable work piece hardness on surface roughness in an end milling using multiple regression and adaptive Neuro fuzzy inference system

Article history: Received July 2 2013 Received in revised format September 7 2013 Accepted November 23 2013 Available online November 26 2013 The aim of this study is to correlate work piece material hardness with surface roughness in prediction studies. The proposed model is for prediction of surface roughness of tool steel materials of hardness 55 HRC to 62 HRC (±2 HRC). The machining experiments are performed under various cutting conditions using work piece of different hardness. The surface roughness of these specimens is measured. The result showed that the influence of work piece material hardness on surface finish is significant for cutting speed and feed in CNC end milling operation. It is also observed that the surface roughness prediction accuracy of Adaptive neuro fuzzy inference system using triangular membership function is better than Gaussian, bell shape membership function and regression analysis. Surface roughness prediction accuracy with material hardness as input parameter is 97.61%. © 2014 Growing Science Ltd. All rights reserved

Time-warped geodesic regression.

We consider geodesic regression with parametric time-warps. This allows for example, to capture saturation effects as typically observed during brain development or degeneration. While highly-flexible models to analyze time-varying image and shape data based on generalizations of splines and polynomials have been proposed recently, they come at the cost of substantially more complex inference. Our focus in this paper is therefore to keep the model and its inference as simple as possible while allowing to capture expected biological variation. We demonstrate that by augmenting geodesic regression with parametric time-warp functions, we can achieve comparable flexibility to more complex models while retaining model simplicity. In addition, the time-warp parameters provide useful information of underlying anatomical changes as demonstrated for the analysis of corpora callosa and rat calvariae. We exemplify our strategy for shape regression on the Grassmann manifold, but note that the method is generally applicable for time-warped geodesic regression.

Face Alignment by Explicit Shape Regression

We present a very efficient, highly accurate, "Explicit Shape Regression" approach for face alignment. Unlike previous regression-based approaches, we directly learn a vectorial regression function...

Real-time facial animation on mobile devices

We present a performance-based facial animation system capable of running on mobile devices at real-time frame rates. A key component of our system is a novel regression algorithm that accurately infers the facial motion parameters from 2D video frames of an ordinary web camera. Compared with the state-of-the-art facial shape regression algorithm [1], which takes a two-step procedure to track facial animations (i.e., first regressing the 3D positions of facial landmarks, and then computing the head poses and expression coefficients), we directly regress the head poses and expression coefficients. This one-step approach greatly reduces the dimension of the regression target and significantly improves the tracking performance while preserving the tracking accuracy. We further propose to collect the training images of the user under different lighting environments, and make use of the data to learn a user-specific regressor, which can robustly handle lighting changes that frequently occur when using mobile devices.

3D shape regression for real-time facial animation

We present a real-time performance-driven facial animation system based on 3D shape regression. In this system, the 3D positions of facial landmark points are inferred by a regressor from 2D video frames of an ordinary web camera. From these 3D points, the pose and expressions of the face are recovered by fitting a user-specific blendshape model to them. The main technical contribution of this work is the 3D regression algorithm that learns an accurate, user-specific face alignment model from an easily acquired set of training data, generated from images of the user performing a sequence of predefined facial poses and expressions. Experiments show that our system can accurately recover 3D face shapes even for fast motions, non-frontal faces, and exaggerated expressions. In addition, some capacity to handle partial occlusions and changing lighting conditions is demonstrated.

Brain shape regression components.

Identifying associations between the shape properties of brain regions, measured from magnetic resonance imaging (MRI), and numerical measures of neurodegenerative disease burden can clarify whether disease processes lead to distinctive spatial patterns of brain atrophy. However, prior methods for identifying such associations between shape and clinical variables either failed to summarize shape patterns into a concise set of summary measurements, or risked failing to discover such associations by extracting summary shape features blinded to the clinical variables. We present a method that overcomes these limitations by directly searching for a small set of linear shape features--shape regression components--that simultaneously account for a large amount of population shape variability and are highly correlated with a numerical clinical variable of interest. When applied to hippocampi of 299 Alzheimer's Disease Neuroimaging Initiative (ADNI) participants, the method identified correlations between hippocampal atrophy and markers of AD pathology and cogniton that were stronger than, and covered a more extended spatial region than, those identified by competing approaches.

Application of residuals from regression of experimental mode shapes to locate multiple crack damage in a simply supported reinforced concrete beam

The objective of the study was to propose an effective, simple and reliable technique to determine the location of triple-crack damage in a simply-supported reinforced concrete beam using the method of mode shape regression. The study required simply supported finite element reinforced concrete beam models to be constructed with one as control and another as the test beam with predetermined triple-cracks along the length of the beam. The technique necessitated the performance of linear and Eigen analyses on the control beam, and nonlinear analysis on the beam with triple-cracks. Residuals obtained from regression of the mode shape using the Chebyshev series rational on the modal frequencies and transformation and application into the fourth order centered finite divided difference formula were used. The use of the regressed mode shapes for the reinforced concrete beam model showed large residuals around the areas of the crack damage. The results showed that the method was successful in determining the locations of the triple cracks and was comparable with other techniques proposed by other researchers in terms of its simplicity and reliability.