Cascaded Regression Framework Research Articles

AMA-Det: Enhancing Shared Head of One-Stage Object Detection With Adaptation, Merging, and Alignment

Feature adaptation(FA) and result alignment(RA) are critical issues in one-stage object detection, since FA amends feature misalignment problem by adapting sampling points to semantically significant locations, and RA corrects result misalignment problem by estimating the localization quality. For FA, the aligned and consistent prediction of sampling points is important. Previous studies directly inherit cascaded “proposal-refine” philosophy from multi-stage detectors and predict sampling points by approximating their minimal external rectangle to ground-truth bounding boxes. This manner generates poorly aligned and consistent sampling points and induces irrelevant features aggregated. Moreover, their sampling points for classification are conventionally generated through the localization branch, without feeding the corresponding features of the classification branch. For RA, previous studies have verified the superiority of utilizing the predicted edge distribution to estimate localization quality; however, their directly regressed distribution is incompatible with the cascaded regression framework. To solve these problems, we firstly propose a focused feature adaptation method by softening the supervision of the proposal points. This method can predict sampling points focused above the assigned objects with excellent alignment and consistency. Subsequently, inner-branch and cross-branch merging were investigated to promote feature sharing from the classification branch. Finally, cascaded distribution-guided result alignment is advanced and verified to predict accurate localization quality. After integrating our proposed adaptation, merging, and alignment, we created AMA-Det with an enhanced shared head, which impressively reaches 43.9 mAP with ResNet50 as the backbone. AMA-Det also achieves a 54.5 mAP by multi-scale testing on MSCOCO test-dev and outperforms all existing CNN-based one-stage counterparts.

Cascade Learning for Driver Facial Monitoring

As a non-invasive method, vision-based driver monitoring aims to identify risky maneuvers for intelligent vehicles and it has gained an increasing interest over recent years. However, most existing methods tend to design models for specific tasks, such as head pose or gaze estimation, which results in redundant models hampering real time applications. Besides, most driver facial monitoring methods ignore the correlation of different tasks. In this work, we propose a unified framework based on cascade learning for simultaneous facial landmark detection and head pose estimation, as well as simultaneous eye center detection and gaze estimation. In particular, built upon the key idea that facial landmark locations and 3D face model parameters are implicitly correlated, we introduce a cascade regression framework to achieve these two tasks simultaneously. After coarsely extracting the driver’s eye region from the detected facial landmarks, we perform cascade regression for simultaneous eye center detection and gaze estimation. Leveraging the power of cascade learning allows our method to alternatively optimize facial landmark detection, head pose estimation, eye center localization, and gaze prediction. The comparison experiments conducted on benchmark datasets of 300-W, GI4E, BU, MPIIGaze, and driving dataset of SHRP2 demonstrate that our proposed method can achieve state-of-the-art performance with robust effectiveness on the real driver monitoring applications.

A Detection of Amblyopia Medical Condition in Biomedical Datasets Using Image Segmentation and Detection Processing

The recent past, the data volume in a media field is growing at a rapid rate, and conventional methods fail to manage such a large volume of data in healthcare systems, biomedical field, medical diagnostic systems etc. The main challenges associated with biomedical computation are the problems associated with management, storage, and analysis on extensive biomedical data. To play a significant role over such extensive data, the machine learning approach provides faster access to medical data with an improved framework. The main objective involves the detection of amblyopia condition from input images and comparing it with conventional image detection methods. The proposed method is examined in terms of detection accuracy, sensitivity, specificity, Hausdorff distance computation and Dice Coefficient. Also, the detection of an Amblyopic or Lazy Eye diseased images is still not prevalent in the field of image segmentation and detection. In this paper, we introduce a framework to process the Amblyopia image datasets using machine learning, and similarity comparison approach. The proposed image processing involves the segmentation of eye images using Recurrent Neural Networks (RNN), and the detection of Amblyopia disease is carried out with Hausdorff Distance computation and Dice coefficient similarity comparison on the segmented image. The initial subset points and threshold values are calculated from a set of 50 normal eye images. A set of 100 Amblyopic diseased image dataset is used for testing the proposed system, out of which 70 images are used for training the system. To evaluate the experimental results shows that proposed method obtains improved detection than existing Deeply-Learned Gaze Shifting Path (DLGSP), Cascade Regression Framework (CRF) and Mobile Iris Recognition System (MIRS) methods. The presence of Hausdorff Distance computation and Dice coefficient similarity comparison is used for reducing the overhead in the proposed method, and this can be used for computing large sets of images.

Cascaded Regression-Based Segmentation of Cardiac CT under Probabilistic Correspondences

The creation of 3D models for cardiac mapping systems is time-consuming, and the models suffer from issues with repeatability among operators. The present study aimed to construct a double-shaped model composed of the left ventricle and left atrium. We developed cascaded-regression-based segmentation software with probabilistic point and appearance correspondence. Group-wise registration of point sets constructs the point correspondence from probabilistic matches, and the proposed method also calculates appearance correspondence from these probabilistic matches. Final point correspondence of group-wise registration constructed independently for three surfaces of the double-shaped model. Stochastic appearance selection of cascaded regression enables the effective construction in the aspect of memory usage and computation time. The two correspondence construction methods of active appearance models were compared in terms of the paired segmentation of the left atrium (LA) and left ventricle (LV). The proposed method segmented 35 cardiac CTs in six-fold cross-validation, and the symmetric surface distance (SSD), Hausdorff distance (HD), and Dice coefficient (DC), were used for evaluation. The proposed method produced 1.88 ± 0.37 mm of LV SSD, 2.25 ± 0.51 mm* of LA SSD, and 2.06 ± 0.34 mm* of the left heart (LH) SSD. Additionally, DC was 80.45% ± 4.27%***, where * p < 0.05, ** p < 0.01, and *** p < 0.001. All p values derive from paired t-tests comparing iterative closest registration with the proposed method. In conclusion, the authors developed a cascaded regression framework for 3D cardiac CT segmentation.

Gradient Shape Model

For years, the so-called Constrained Local Model (CLM) and its variants have been the gold standard in face alignment tasks. The CLM combines an ensemble of local feature detectors whose locations are regularized by a shape model. Fitting such a model typically consists of an exhaustive local search using the detectors and a global optimization that finds the CLM’s parameters that jointly maximize all the responses. However, one major drawback of CLMs is the inefficiency of the local search, which relies on a large amount of expensive convolutions. This paper introduces the Gradient Shape Model (GSM), a novel approach that addresses this limitation. We are able to align a similar CLM model without the need for any convolutions at all. We also use true analytical gradient and Hessian matrices, which are easy to compute, instead of their approximations. Our formulation is very general, allowing an optional 3D shape term to be seamlessly included. Additionally, we expand the GSM formulation through a cascade regression framework. This revised technique allows a substantially reduction in the complexity/dimensionality of the data term, making it possible to compute a denser, more accurate, regression step per cascade level. Experiments in several standard datasets show that our proposed models perform faster than state-of-the-art CLMs and better than recent cascade regression approaches.

Coupled cascade regression from real and synthesized faces for simultaneous landmark detection and head pose estimation

The existing approaches usually perform facial landmark detection and head pose estimation independently and sequentially, ignoring their coupled relations. We introduce a unified framework, named coupled cascade regression (CCR), for simultaneous facial landmark detection and head pose estimation. Based on the cascade regression framework, we propose to learn two separate regressors to update the landmark locations and three-dimensional (3D) face model parameters at each cascade level. To capture the coupled relations of the landmark locations and head pose, we further apply the 3D face projection model to refine the prediction results in each cascade iteration and make them consistent. CCR can leverage both the learning methods and the projection model to simultaneously perform facial landmark detection and pose estimation to enhance the performances of both tasks. We also propose to learn the cascade regressors from the combination of real and synthesized face images to solve the problem of limited variations in head pose for training. Experimental results on Helen, labeled face parts in the wild, 300-W, and Boston University datasets show that our proposed CCR method outperforms other conventional methods both for landmark detection and head pose estimation.

Feedback cascade regression model for face alignment

Face alignment has made great progress in recent years and the cascade regression framework is one of the main contributors. However, the performance of this framework is unsatisfactory on heavily occluded faces or those far from the frontal pose. This is because regression is sensitive to hidden landmarks and unified initialisation can often lead to the method falling into local minima. The authors propose a new pipeline of salient‐to‐inner‐to‐all to progressively compute the locations of landmarks. Additionally, a feedback process is utilised to improve the robustness of regression. They bring out a pose‐invariant shape retrieval method to generate the discriminative initialisation. Experiments are performed on two benchmarks, and the experimental results demonstrate that the proposed method has a considerable improvement on the cascade regression model, and achieves favourable results compared with the state‐of‐the‐art deep learning‐based methods.

Cascaded regression using landmark displacement for 3D face reconstruction

This paper proposes a novel model fitting algorithm for 3D facial expression reconstruction from a single image. Face expression reconstruction from a single image is a challenging task in computer vision. Most state-of-the-art methods fit the input image to a 3D Morphable Model (3DMM). These methods need to solve a stochastic problem and cannot deal with expression and pose variations. To solve this problem, we adopt a 3D face expression model and use a combined feature which is robust to scale, rotation and different lighting conditions. The proposed method applies a cascaded regression framework to estimate parameters for the 3DMM. 2D landmarks are detected and used to initialize the 3D shape and mapping matrices. In each iteration, residues between the current 3DMM parameters and the ground truth are estimated and then used to update the 3D shapes. The mapping matrices are also calculated based on the updated shapes and 2D landmarks. HOG features of the local patches and displacements between 3D landmark projections and 2D landmarks are exploited. Compared with existing methods, the proposed method is robust to expression and pose changes and can reconstruct higher fidelity 3D face shape.

Joint 3-D Shape Estimation and Landmark Localization From Monocular Cameras of Intelligent Vehicles

3-D reconstruction is at the core for many driving applications of Internet of Intelligent Vehicles. Previous works on reconstruction of a 3-D point shape commonly use a two-step framework. Precisely localizing a series of feature points in an image is performed on the first step. Then the second procedure attempts to fit the 3-D data to the observations to get the real 3-D shape. Such an approach has high time consumption, and easily gets stuck into local minimum. To address this problem, we propose a method to jointly estimate the global 3-D geometric structure of car and localize 2-D landmarks from a single viewpoint image. First, we represent the 3-D shape with a set of predefined shape bases, while parametrizing it by the coefficients of the linear combination of them. Second, we adopt a cascaded regression framework to regress the global shape encoded by the prior bases, by jointly minimizing the appearance and shape fitting differences. The position fitting item can help cope with the description ambiguity of local appearance, and provide more information for 3-D reconstruction. We apply the proposed approach on a multiview car dataset. Experimental results demonstrate favorable improvements on pose estimation and shape prediction, compared with some previous methods.

Dual-Domain Cascaded Regression for Synthesizing 7T from 3T MRI.

Due to the high cost and low accessibility of 7T magnetic resonance imaging (MRI) scanners, we propose a novel dual-domain cascaded regression framework to synthesize 7T images from the routine 3T images. Our framework is composed of two parallel and interactive multi-stage regression streams, where one stream regresses on spatial domain and the other regresses on frequency domain. These two streams complement each other and enable the learning of complex mappings between 3T and 7T images. We evaluated the proposed framework on a set of 3T and 7T images by leave-one-out cross-validation. Experimental results demonstrate that the proposed framework generates realistic 7T images and achieves better results than state-of-the-art methods.

A Functional Regression Approach to Facial Landmark Tracking.

Linear regression is a fundamental building block in many face detection and tracking algorithms, typically used to predict shape displacements from image features through a linear mapping. This paper presents a Functional Regression solution to the least squares problem, which we coin Continuous Regression, resulting in the first real-time incremental face tracker. Contrary to prior work in Functional Regression, in which B-splines or Fourier series were used, we propose to approximate the input space by its first-order Taylor expansion, yielding a closed-form solution for the continuous domain of displacements. We then extend the continuous least squares problem to correlated variables, and demonstrate the generalisation of our approach. We incorporate Continuous Regression into the cascaded regression framework, and show its computational benefits for both training and testing. We then present a fast approach for incremental learning within Cascaded Continuous Regression, coined iCCR, and show that its complexity allows real-time face tracking, being 20 times faster than the state of the art. To the best of our knowledge, this is the first incremental face tracker that is shown to operate in real-time. We show that iCCR achieves state-of-the-art performance on the 300-VW dataset, the most recent, large-scale benchmark for face tracking.

Salient-points-guided face alignment

Regression-based face alignment approach is fast and accurate but is always limited by the initial face. Aim at limitation of initialization in regression methods, this paper presents a novel two-stage framework named salient-points-guided face alignment. In first stage, we use cascade regression framework to train a salient points (eye centers, nose, mouth corners) localization model. Then the salient points information is used as a guidance for searching the similar faces from training set. In second stage, leveraging the similar faces to generate the initial face for all points regression. In order to give more comprehensive comparison, a new evaluation metric is proposed. Considering the global distance between estimated face and ground-truth, the new evaluation metric is defined as sum of the global distance and the widely used average point-to-point distance. The results show that our approach can achieve state-of-the-art performance (12% higher than the human performance on COFW) and the new evaluation metric is more reasonable.

Supervised coordinate descent method with a 3D bilinear model for face alignment and tracking

AbstractFace alignment and tracking play important roles in facial performance capture. Existing data‐driven methods for monocular videos suffer from large variations of pose and expression. In this paper, we propose an efficient and robust method for this task by introducing a novel supervised coordinate descent method with 3D bilinear representation. Instead of learning the mapping between the whole parameters and image features directly with a cascaded regression framework in current methods, we learn individual sets of parameters mappings separately step by step by a coordinate descent mean. Because different parameters make different contributions to the displacement of facial landmarks, our method is more discriminative to current whole‐parameter cascaded regression methods. Benefiting from a 3D bilinear model learned from public databases, the proposed method can handle the head pose changes and extreme expressions out of plane better than other 2D‐based methods. We present the reliable result of face tracking under various head poses and facial expressions on challenging video sequences collected online. The experimental results show that our method outperforms state‐of‐art data‐driven methods.

A joint cascaded framework for simultaneous eye detection and eye state estimation

Eye detection and eye state (close/open) estimation are important for a wide range of applications, including iris recognition, visual interaction and driver fatigue detection. Current work typically performs eye detection first, followed by eye state estimation by a separate classifier. Such an approach fails to capture the interactions between eye location and its state. In this paper, we propose a method for simultaneous eye detection and eye state estimation. Based on a cascade regression framework, our method iteratively estimates the location of the eye and the probability of the eye being occluded by eyelid. At each iteration of cascaded regression, image features from the eye center as well as contextual image features from eyelid and eye corners are jointly used to estimate the eye position and openness probability. Using the eye openness probability, the most likely eye state can be estimated. Since it requires large number of facial images with labeled eye related landmarks, we propose to combine the real and synthetic images for training. It further improves the performance by utilizing this learning-by-synthesis method. Evaluations of our method on benchmark databases such as BioID and Gi4E database as well as on real world driving videos demonstrate its superior performance comparing to state-of-the-art methods for both eye detection and eye state estimation.

Facial point localization via neural networks in a cascade regression framework

Facial point detection gains an increasing importance in computer vision as it plays a vital role in several applications such as facial expression recognition and human behavior analysis. In this work, we propose an approach to locate 49 facial points via neural networks in a cascade regression fashion. The localization process starts by detecting the face, followed by a face cropping refinement task and lastly arriving at the facial point location through five cascades of regressors. In particular, we perform a guided initialization using holistic features extracted from the entire face patch. Then, the points location is refined in the next four cascades using local features extracted from patches enclosing the prior estimates of the points. The generalization capability was improved by performing feature selection at each cascade. By evaluating our approach on samples gathered from four challenging databases, we achieved a location average error for each point ranging between 0.72 % and 1.57 % of the face width. The proposed approach was further evaluated according to the 300-w challenge, where we achieved competitive results to those obtained by state-of-the-art approaches and commercial software packages. Moreover, our approach showed better generalization capability. Finally, we validated the proposed enhancements by studying the impact of several factors on the point localization accuracy.

Dense 3D face alignment from 2D video for real-time use

To enable real-time, person-independent 3D registration from 2D video, we developed a 3D cascade regression approach in which facial landmarks remain invariant across pose over a range of approximately 60°. From a single 2D image of a person's face, a dense 3D shape is registered in real time for each frame. The algorithm utilizes a fast cascade regression framework trained on high-resolution 3D face-scans of posed and spontaneous emotion expression. The algorithm first estimates the location of a dense set of landmarks and their visibility, then reconstructs face shapes by fitting a part-based 3D model. Because no assumptions are required about illumination or surface properties, the method can be applied to a wide range of imaging conditions that include 2D video and uncalibrated multi-view video. The method has been validated in a battery of experiments that evaluate its precision of 3D reconstruction, extension to multi-view reconstruction, temporal integration for videos and 3D head-pose estimation. Experimental findings strongly support the validity of real-time, 3D registration and reconstruction from 2D video. The software is available online at http://zface.org.

Transferred Deep Convolutional Neural Network Features for Extensive Facial Landmark Localization

Features are crucial for extensive facial landmark localization (EFLL), while deep convolutional neural network (DCNN) features lead to breakthroughs in diverse visual recognition tasks. However, there is little study of DCNN features for EFLL, mainly because less labeled data with extensive facial landmarks are available. In this letter, we employ transfer learning to overcome this limitation, and utilize DCNN for EFLL. We concentrate the power of DCNN on feature learning within a cascaded-regression framework (CRF). We present three transfer methods, which show the capacity of DCNN as a generic feature extractor, and the benefit of fine-tuning. The proposed specific fine-tuning method for cascaded regression, named cascade transfer, achieves competitive accuracy with state-of-the-art methods on the 300-W challenge dataset.

Dense 3D Face Alignment from 2D Videos in Real-Time.

To enable real-time, person-independent 3D registration from 2D video, we developed a 3D cascade regression approach in which facial landmarks remain invariant across pose over a range of approximately 60 degrees. From a single 2D image of a person's face, a dense 3D shape is registered in real time for each frame. The algorithm utilizes a fast cascade regression framework trained on high-resolution 3D face-scans of posed and spontaneous emotion expression. The algorithm first estimates the location of a dense set of markers and their visibility, then reconstructs face shapes by fitting a part-based 3D model. Because no assumptions are required about illumination or surface properties, the method can be applied to a wide range of imaging conditions that include 2D video and uncalibrated multi-view video. The method has been validated in a battery of experiments that evaluate its precision of 3D reconstruction and extension to multi-view reconstruction. Experimental findings strongly support the validity of real-time, 3D registration and reconstruction from 2D video. The software is available online at http://zface.org.