Large Pose Variations Research Articles

Pose-Invariant Face Recognition via RGB-D Images.

Three-dimensional (3D) face models can intrinsically handle large pose face recognition problem. In this paper, we propose a novel pose-invariant face recognition method via RGB-D images. By employing depth, our method is able to handle self-occlusion and deformation, both of which are challenging problems in two-dimensional (2D) face recognition. Texture images in the gallery can be rendered to the same view as the probe via depth. Meanwhile, depth is also used for similarity measure via frontalization and symmetric filling. Finally, both texture and depth contribute to the final identity estimation. Experiments on Bosphorus, CurtinFaces, Eurecom, and Kiwi databases demonstrate that the additional depth information has improved the performance of face recognition with large pose variations and under even more challenging conditions.

Dual Sparse Constrained Cascade Regression for Robust Face Alignment.

Localizing facial landmarks is a fundamental step in facial image analysis. However, the problem continues to be challenging due to the large variability in expression, illumination, pose, and the existence of occlusions in real-world face images. In this paper, we present a dual sparse constrained cascade regression model for robust face alignment. Instead of using the least-squares method during the training process of regressors, sparse constraint is introduced to select robust features and compress the size of the model. Moreover, sparse shape constraint is incorporated between each cascade regression, and the explicit shape constraints are able to suppress the ambiguity in local features. To improve the model's adaptation to large pose variation, face pose is estimated by five fiducial landmarks located by deep convolutional neuron network, which is used to adaptively design the cascade regression model. To the best of our best knowledge, this is the first attempt to fuse explicit shape constraint (sparse shape constraint) and implicit context information (sparse feature selection) for robust face alignment in the framework of cascade regression. Extensive experiments on nine challenging wild data sets demonstrate the advantages of the proposed method over the state-of-the-art methods.

Register-My-Face: a tool to register three-dimensional face images

Efficient registration across pose is the most challenging research area for accurate recognition of human face images. Authors have discussed a tool that has been developed for registration of face images across poses using the nose tip of the face images. The nose tip has been considered here because of its stability in situations such as variations in pose, expression, etc. The aim of this investigation is to develop a face registration tool called “Register-My-Face” with a working methodology in all the three directions, namely yaw, pitch, and roll. This tool has been developed for three-dimensional (3-D) face registration, which is inspired by analyzing the “depth values” of face range images. The registration of the face is done using a geometrical technique which is based on computing the corresponding rotation in three orthogonal directions. The advantages of the designed tool are that it does not need any training phase for accurate detection of the nose tip, and this method can handle large pose variations, including 90 deg pose variations about the Y-axis in both the positive and negative directions. The method that has been followed to develop this tool is also independent of facial expression, occlusion and illumination variations. Moreover, it quickly detects the nose tip because it does not need to process the entire face surface, but only requires the isolated nose region. The tool has been integrated with three different databases; GavabDB, Bosphorus, and Frav3D, and the investigation highlights the robustness of the tool. Additionally, for exploring the performance of the tool, a SIMULINK model for hardware interface is also developed with a discrete solver and is tested on two different configuration setups and executed in two different execution modes with two simulation stop timings 10.0 and 1.0. This model can proceed according to the algorithm with a minimum of 6.640 s to register an unregistered raw 3-D face scan input image from the Frav3D database. Accuracies of the nose region of 98.87%, 94.44%, and 98.08% for the Frav3D, GavabDB, and Bosphorus databases, respectively, are observed. For nose tip detection, the success rates are 98.91% for the Frav3D database, 98.74% for the GavabDB database, and 96.03% for the Bosphorus database. Based on the success rate of nose tip detection, the registration process is implemented on three databases. Registration accuracy, computed between a neutral and the registered range face image for the Frav3D database is 87.5% for GavabDB and 89.87% for Bosphorus database, and the rate of success is 70.23%.

Robust multi-object tracking using deep learning framework

Object appearance representation is crucial to any visual tracker. In order to improve the appearance representation, we propose a deep compact and high-level appearance representation applied to a multi-object tracking algorithm, which is called Deep Multi-object Tracking. In this paper, we adopt the deep learning framework to offline obtain generic image features with auxiliary natural images and online fine-tune our Deep Multi-object Tracking system to adapt to appearance changes of the moving objects. Besides, we have fully considered the temporal information denoting the dynamic duration time of each object. Based on the temporal information and particle filter, our Deep Multi-object Tracking algorithm can effectively generate an online learning and updating model to form a discriminative appearance scheme to achieve successful multi-object tracking. Experiments show that the proposed Deep Multi-object Tracking performs well both indoor and outdoor, where the objects undergo large pose, scale, occlusion, and illumination variations in complex scenes including abnormal ones.

A robust tracking method with adaptive local spatial sparse representation

In this paper, a robust visual tracking method is proposed based on local spatial sparse representation. In the proposed approach, the learned target template is sparsely and compactly expressed by forming local spatial and trivial samples dynamically. An adaptive multiple subspaces appearance model is developed to describe the target appearance and construct the candidate target templates during the tracking process. An effective selection strategy is then employed to select the optimal sparse solution and locate the target accurately in the next frame. The experimental results have demonstrated that our method can perform well in the complex and noisy visual environment, such as heavy occlusions, dramatic illumination changes, and large pose variations in the video. Copyright © 2015 John Wiley & Sons, Ltd.

Articulated Non-Rigid Point Set Registration for Human Pose Estimation from 3D Sensors.

We propose a generative framework for 3D human pose estimation that is able to operate on both individual point sets and sequential depth data. We formulate human pose estimation as a point set registration problem, where we propose three new approaches to address several major technical challenges in this research. First, we integrate two registration techniques that have a complementary nature to cope with non-rigid and articulated deformations of the human body under a variety of poses. This unique combination allows us to handle point sets of complex body motion and large pose variation without any initial conditions, as required by most existing approaches. Second, we introduce an efficient pose tracking strategy to deal with sequential depth data, where the major challenge is the incomplete data due to self-occlusions and view changes. We introduce a visible point extraction method to initialize a new template for the current frame from the previous frame, which effectively reduces the ambiguity and uncertainty during registration. Third, to support robust and stable pose tracking, we develop a segment volume validation technique to detect tracking failures and to re-initialize pose registration if needed. The experimental results on both benchmark 3D laser scan and depth datasets demonstrate the effectiveness of the proposed framework when compared with state-of-the-art algorithms.

Mixed-norm sparse representation for multi view face recognition

Face recognition with multiple views is a challenging research problem. Most of the existing works have focused on extracting shared information among multiple views to improve recognition. However, when the pose variation is too large or missing, ‘shared information’ may not be properly extracted, leading to poor recognition results. In this paper, we propose a novel method for face recognition with multiple view images to overcome the large pose variation and missing pose issue. By introducing a novel mixed norm, the proposed method automatically selects candidates from the gallery to best represent a group of highly correlated face images in a query set to improve classification accuracy. This mixed norm combines the advantages of both sparse representation based classification (SRC) and joint sparse representation based classification (JSRC). A trade off between the ℓ1-norm from SRC and ℓ2,1-norm from JSRC is introduced to achieve this goal. Due to this property, the proposed method decreases the influence when a face image is unseen and has large pose variation in the recognition process. And when some face images with a certain degree of unseen pose variation appear, this mixed norm will find an optimal representation for these query images based on the shared information induced from multiple views. Moreover, we also address an open problem in robust sparse representation and classification which is using ℓ1-norm on the loss function to achieve a robust solution. To solve this formulation, we derive a simple, yet provably convergent algorithm based on the powerful alternative directions method of multipliers (ADMM) framework. We provide extensive comparisons which demonstrate that our method outperforms other state-of-the-arts algorithms on CMU-PIE, Yale B and Multi-PIE databases for multi-view face recognition.

High-Resolution Face Verification Using Pore-Scale Facial Features

Face recognition methods, which usually represent face images using holistic or local facial features, rely heavily on alignment. Their performances also suffer a severe degradation under variations in expressions or poses, especially when there is one gallery per subject only. With the easy access to high-resolution (HR) face images nowadays, some HR face databases have recently been developed. However, few studies have tackled the use of HR information for face recognition or verification. In this paper, we propose a pose-invariant face-verification method, which is robust to alignment errors, using the HR information based on pore-scale facial features. A new keypoint descriptor, namely, pore-Principal Component Analysis (PCA)-Scale Invariant Feature Transform (PPCASIFT)-adapted from PCA-SIFT-is devised for the extraction of a compact set of distinctive pore-scale facial features. Having matched the pore-scale features of two-face regions, an effective robust-fitting scheme is proposed for the face-verification task. Experiments show that, with one frontal-view gallery only per subject, our proposed method outperforms a number of standard verification methods, and can achieve excellent accuracy even the faces are under large variations in expression and pose.

Using an improved SIFT algorithm and fuzzy closed-loop control strategy for object recognition in cluttered scenes.

Partial occlusions, large pose variations, and extreme ambient illumination conditions generally cause the performance degradation of object recognition systems. Therefore, this paper presents a novel approach for fast and robust object recognition in cluttered scenes based on an improved scale invariant feature transform (SIFT) algorithm and a fuzzy closed-loop control method. First, a fast SIFT algorithm is proposed by classifying SIFT features into several clusters based on several attributes computed from the sub-orientation histogram (SOH), in the feature matching phase only features that share nearly the same corresponding attributes are compared. Second, a feature matching step is performed following a prioritized order based on the scale factor, which is calculated between the object image and the target object image, guaranteeing robust feature matching. Finally, a fuzzy closed-loop control strategy is applied to increase the accuracy of the object recognition and is essential for autonomous object manipulation process. Compared to the original SIFT algorithm for object recognition, the result of the proposed method shows that the number of SIFT features extracted from an object has a significant increase, and the computing speed of the object recognition processes increases by more than 40%. The experimental results confirmed that the proposed method performs effectively and accurately in cluttered scenes.

Tracking with spatial constrained coding

A video tracking method based on spatial constrained coding (SCC) is proposed in this study. To characterise local image structure information, the dense scale-invariant feature transform (SIFT) descriptor is extracted for each pixel in the image. The proposed tracking method uses SCC model which adopts a new constrained strategy – weighted code, which is achieved by considering the sum of the weighted codes based on grey values of neighbouring pixels and distances between them. The proposed model is able to obtain robust code of corresponding pixels in the frames of complex scenes by taking spatial information into account, which enhances the stability of coding and makes the tracker more robust for object tracking. Twelve challenging sequences involving partial or full occlusion, large pose variation and drastic illumination change are chosen to test the proposed method. The experimental results show the proposed method performs excellent in comparison with other previously proposed trackers.

EAR BIOMETRICS AND SPARSE REPRESENTATION BASED ON SMOOTHED l0 NORM

Ear biometrics attracted the attention of researchers in computer vision and machine learning for its use in many applications. In this paper, we present a fully automated system for recognition from ear images based upon sparse representation. In sparse representation, extracted features from the training data is used to develop a dictionary. Classification is achieved by representing the extracted features of the test data as a linear combination of entries in the dictionary. In fact, there are many solutions for this problem and the goal is to find the sparsest solution. We use a relatively new algorithm named smoothed l0 norm to find the sparsest solution and Gabor wavelet features are used for building the dictionary. Furthermore, we expand the proposed approach for gender classification from ear images. Several researches have addressed this issue based on facial images. We introduce a novel approach based on majority voting for gender classification. Experimental results conducted on the University of Notre Dame (UND) collection J data set, containing large appearance, pose, and lighting variations, resulted in a gender classification rate of 89.49%. Furthermore, the proposed method is evaluated on the WVU data set and classification rates for different view angles are presented. Results show improvement and great robustness in gender classification over existing methods.

Uncooperative pose estimation with a LIDAR-based system

This paper aims at investigating the performance of a LIDAR-based system for pose determination of uncooperative targets. This problem is relevant to both debris removal and on-orbit servicing missions, and requires the adoption of suitable electro-optical sensors on board of a chaser platform, as well as model-based techniques for target detection and pose estimation. In this paper, a three dimensional approach is pursued in which the point cloud generated by a LIDAR is exploited for pose estimation. Specifically, the condition of close proximity flight to a large debris is considered, in which the relative motion determines a large variation of debris appearance and coverage in the sensor field of view, thus producing challenging conditions for pose estimation. A customized three dimensional Template Matching approach is proposed for fast and reliable pose initial acquisition, while pose tracking is carried out with an Iterative Closest Point algorithm exploiting different measurement-model matching techniques. Specific solutions are envisaged to speed algorithm convergence and limit the size of the point clouds used for pose initial acquisition and tracking to allow autonomous on-board operation. To investigate proposed approach effectiveness and achievable pose accuracy, a numerical simulation environment is developed implementing realistic debris geometry, debris-chaser close-proximity flight, and sensor operation. Results demonstrate algorithm capability of operating with sparse point clouds and large pose variations, while achieving sub-degree and sub-centimeter accuracy in relative attitude and position, respectively.

Face recognition from a single registered image for conference socializing

Scientific conferences are primary venues for connecting with and forming relationships with fellow researchers and scientists. Thus, over the course of a conference participants often take advantage of the many opportunities to network. In this setting, it is desirable to quickly recognize the identity of the persons we see and wish to meet. In particular, it could be embarrassing to not recognize a prominent researcher. In this paper, we investigate a novel face recognition framework that is applicable to conference socialization scenarios. In the proposed framework, only frontal images are used as training images; and face recognition is possible from an arbitrary view of a subject. Our system prototype assumes that the conference participants have uploaded a frontal photo during the registration process. At the conference, the identity of a person can be recognized from a picture, taken from an arbitrary angle with a standard mobile phone. Our experimental results indicate that the proposed framework is robust to possible large pose variations between the non-frontal image captured impromptu and the training image of the same person. Experiments based upon standard face dataset and real conference socializing datasets are conducted to test the effectiveness of the proposed techniques.

Inter-frame constrained coding based on superpixel for tracking

A video tracking method based on superpixel with inter-frame constrained coding is proposed in this paper. The 3-D CIE Lab color feature is extracted at each superpixel to characterize local image information. Based on the color feature, the superpixel-based coding model is achieved between the adjacent frames for correctly tracking object. The proposed tracking method considers the interaction of corresponding superpixels between the adjacent frames of complex scenes, which enhances the stability of encoding. Due to the update of codebook and classifier parameter, the proposed method is robust for long-term object tracking. We test the proposed method on 15 challenging sequences involving drastic illumination change, partial or full occlusion, and large pose variation. The proposed method shows excellent performance in comparison with eight previously proposed trackers.

Video Image Object Tracking Algorithm based on Improved Principal Component Analysis

Since the existing object tracking algorithms are very difficult to adapt the object appearance changes caused by illumination changes, large pose variations, and partial or full occlusions, an object tracking algorithm based on two-dimensional principal component analysis (2DPCA) and sparse-representation is proposed in this paper. The tracking algorithm adopts 2DPCA and sparse-representation to establish object appearance model. In order to reduce dimension of object template, incremental subspace updating algorithm is introduced to online update the object template, reduce the requirement of memory space and enhance the precision of object appearance description. Experimental results show the proposed algorithm is robust for image illumination variance and object partial occlusion.

3D-MRF BASED VIDEO TRACKING IN THE COMPRESSED DOMAIN

Object tracking is an interesting and needed procedure for many real time applications. But it is a challenging one, because of the presence of challenging sequences with abrupt motion, drastic illumination change, large pose variation, occlusion, cluttered background and also the camera shake. This paper presents a novel method of object tracking by using the algorithms spatiotemporal Markov random field (STMRF) and online discriminative feature selection (ODFS), which overcome the above mentioned problems and provide a better tracking process. This method is also capable of tracking multiple objects in video sequence even in the presence of an object interactions and occlusions that achieves better results with real time performance.

Visual Tracking via Online Nonnegative Matrix Factorization

In visual tracking, holistic and part-based representations are both popular choices to model target appearance. The former is known for great efficiency and convenience, while the latter for robustness against local appearance or shape variations. Based on nonnegative matrix factorization (NMF), we propose a novel visual tracker that takes advantage of both groups. The idea is to model the target appearance by a nonnegative combination of nonnegative components learned from examples observed in previous frames. To adjust NMF to the tracking context, we include sparsity and smoothness constraints in addition to the nonnegativity one. Furthermore, an online iterative learning algorithm, together with a proof of convergence, is proposed for efficient model updating. Putting these ingredients together with a particle filter framework, the proposed tracker, constrained online nonnegative matrix factorization (CONMF), achieves robustness to challenging appearance variations and nontrivial deformations while running in real time. We evaluate the proposed tracker on various benchmark sequences containing targets undergoing large variations in scale, pose, or illumination. The robustness and efficiency of CONMF is validated in comparison with several state-of-the-art trackers.

An efficient approach for face recognition based on common eigenvalues

In this paper, a simple technique is proposed for face recognition among many human faces. It is based on the polynomial coefficients, covariance matrix and algorithm on common eigenvalues. The main advantage of the proposed approach is that the identification of similarity between human faces is carried out without computing actual eigenvalues and eigenvectors. A symmetric matrix is calculated using the polynomial coefficients-based companion matrices of two compared images. The nullity of a calculated symmetric matrix is used as similarity measure for face recognition. The value of nullity is very small for dissimilar images and distinctly large for similar face images. The feasibility of the propose approach is demonstrated on three face databases, i.e., the ORL database, the Yale database B and the FERET database. Experimental results have shown the effectiveness of the proposed approach for feature extraction and classification of the face images having large variation in pose and illumination.

Vision-based action recognition of earthmoving equipment using spatio-temporal features and support vector machine classifiers

Video recordings of earthmoving construction operations provide understandable data that can be used for benchmarking and analyzing their performance. These recordings further support project managers to take corrective actions on performance deviations and in turn improve operational efficiency. Despite these benefits, manual stopwatch studies of previously recorded videos can be labor-intensive, may suffer from biases of the observers, and are impractical after substantial period of observations. This paper presents a new computer vision based algorithm for recognizing single actions of earthmoving construction equipment. This is particularly a challenging task as equipment can be partially occluded in site video streams and usually come in wide variety of sizes and appearances. The scale and pose of the equipment actions can also significantly vary based on the camera configurations. In the proposed method, a video is initially represented as a collection of spatio-temporal visual features by extracting space–time interest points and describing each feature with a Histogram of Oriented Gradients (HOG). The algorithm automatically learns the distributions of the spatio-temporal features and action categories using a multi-class Support Vector Machine (SVM) classifier. This strategy handles noisy feature points arisen from typical dynamic backgrounds. Given a video sequence captured from a fixed camera, the multi-class SVM classifier recognizes and localizes equipment actions. For the purpose of evaluation, a new video dataset is introduced which contains 859 sequences from excavator and truck actions. This dataset contains large variations of equipment pose and scale, and has varied backgrounds and levels of occlusion. The experimental results with average accuracies of 86.33% and 98.33% show that our supervised method outperforms previous algorithms for excavator and truck action recognition. The results hold the promise for applicability of the proposed method for construction activity analysis.

3D Face Recognition under Expressions, Occlusions, and Pose Variations

We propose a novel geometric framework for analyzing 3D faces, with the specific goals of comparing, matching, and averaging their shapes. Here we represent facial surfaces by radial curves emanating from the nose tips and use elastic shape analysis of these curves to develop a Riemannian framework for analyzing shapes of full facial surfaces. This representation, along with the elastic Riemannian metric, seems natural for measuring facial deformations and is robust to challenges such as large facial expressions (especially those with open mouths), large pose variations, missing parts, and partial occlusions due to glasses, hair, and so on. This framework is shown to be promising from both--empirical and theoretical--perspectives. In terms of the empirical evaluation, our results match or improve upon the state-of-the-art methods on three prominent databases: FRGCv2, GavabDB, and Bosphorus, each posing a different type of challenge. From a theoretical perspective, this framework allows for formal statistical inferences, such as the estimation of missing facial parts using PCA on tangent spaces and computing average shapes.