Large Pose Variations Research Articles

LiDAR-Based Non-Cooperative Tumbling Spacecraft Pose Tracking by Fusing Depth Maps and Point Clouds.

How to determine the relative pose between the chaser spacecraft and the high-speed tumbling target spacecraft at close range, which is an essential step in space proximity missions, is very challenging. This paper proposes a LiDAR-based pose tracking method by fusing depth maps and point clouds. The key point is to estimate the roll angle variation in adjacent sensor data by using the line detection and matching in depth maps. The simplification of adaptive voxelized grid point cloud based on the real-time relative position is adapted in order to satisfy the real-time requirement in the approaching process. In addition, the Iterative Closest Point algorithm is used to align the simplified sparse point cloud with the known target model point cloud in order to obtain the relative pose. Numerical experiments, which simulate the typical tumbling motion of the target and the approaching process, are performed to demonstrate the method. The experimental results show that the method has capability of estimating the real-time 6-DOF relative pose and dealing with large pose variations.

Scaled and oriented object tracking using ensemble of multilayer perceptrons

Major challenging problems in the field of moving object tracking are to handle changing in scale and orientation, background clutter and large variation in pose with occlusion. This article presents an algorithm to track moving object under such complex environment. Here, a discriminative model based on an ensemble of multilayer perceptrons (MLPs) is proposed to detect object from its cluttered background. Orientation and enhanced scale of the detected object is estimated using binary moments. Here, the problem of object tracking is posed as a constrained optimization with respect to location, scale and orientation of the object. Two different heuristics based on support value and confidence score are proposed to reduce drift and to detect full occlusion. Three benchmark datasets are considered for the experimental purpose and the proposed algorithm attains state-of-the-art performance under various conditions.

Face Alignment Based on Two-Stage Localization

The traditional cascade regression model could easily drop into local optimum when the initial localization is far from real shape.This paper proposesa face alignment scheme based ontwo-stage localization. In theface detection stage, the multi-task CNN is adopted to get the information oflandmarks and poseas the initial localization for second stage. In the second stage, the cascade regression based on random forest is implemented for face alignment, and the localization from first stage is used as the initial coarse localization for regression model to solve the problem of initialization sensitivity. The scheme can achieve excellent performance even in the case of large pose variations and occlusion. Compared with the traditional method, experiments show that our algorithm can get state-of-art resultsin real-time on benchmark of 300W dataset.

Context multi-task visual object tracking via guided filter

In this paper, we formulate particle filter based tracking as a multi-task sparse learning problem that exploits context information. The target and context information which modeled as linear combinations of principal component analysis (PCA) basis is formed as dictionary templates. The target is treated as the guidance and the sampling examples are filtered depending on the similarity between the target and each input. The edge preserving smoothing property of the guided filter is a key factor for object tracking. First, valuable candidates can be selected and the inaccurate candidates become blurry. Therefore, the guided filter can help to distinguish the target from numerous candidates easily. Second, partial occluded target can be recovered by the filtering process via the guidance image. Thus, the drifting problem can be alleviated. Then multi-task sparse learning is employed to learn the target and context information. The proposed learning problem is efficiently solved using an alternating direction method of multipliers (ADMM) method that yields a sequence of closed form updates. We test our tracker on challenging video sequences that involve drastic illumination changes, large pose variations, and heavy occlusions. Experimental results show that our tracker consistently outperforms state-of-the-art trackers.

Multiple face tracking and recognition with identity-specific localized metric learning

While face recognition has been intensively studied in the literature, there are only a few attempts on visual recognition of multiple faces simultaneously in videos, which has potential applications in practical video surveillance. In this paper, we address the problem of visual recognition and tracking of multiple faces in real-world videos involving large pose variation and occlusion. Instead of recognizing individual face independently, we introduce the constraints of inter-frame temporal smoothness and within-frame identity exclusivity on multiple faces in videos, and model the tasks of multiple face recognition (MFR) and multiple face tracking (MFT) jointly in an alternative optimization framework. We show this joint formulation for two different tasks leads to significantly improved MFR accuracy. Specifically, as appearance matching for face instances over consecutive frames plays a critical role in MFT, we propose an identity-specific metric learning method with a part-based object representation to learn a localized transformation for each face subject in an online manner, under which face instances of the same subject over consecutive frames are pulled as close as possible, while those of different subjects are pushed far away. Empirically, we evaluate our method on several MFR sequences against baselines, and the results demonstrate that our method can achieve improved accuracy performance in various challenging recognition scenarios.

Multi-task convolution network for face alignment

In order to solve face alignment more effectively, we proposed a multi-task deep convolution network for face alignment, which achieves good performance even in the case of large pose variations and severe occlusion. Instead of dealing with face alignment as a single task, we jointly trained the auxiliary task of pose estimation together with face alignment to guide the distribution of facial points. By doing so we are able to 1) avoid trapping in the local optimum due to the inaccurate face boxes, 2) improve the robustness in dealing with faces with pose variation and severe occlusion. Compared with the traditional methods, our method also improves the accuracy by providing better initialization instead of mean shape. Extensive experiments show that our method has great performance on various benchmarks.

Robust Face Recognition With Kernelized Locality-Sensitive Group Sparsity Representation.

In this paper, a novel joint sparse representation method is proposed for robust face recognition. We embed both group sparsity and kernelized locality-sensitive constraints into the framework of sparse representation. The group sparsity constraint is designed to utilize the grouped structure information in the training data. The local similarity between test and training data is measured in the kernel space instead of the Euclidian space. As a result, the embedded nonlinear information can be effectively captured, leading to a more discriminative representation. We show that, by integrating the kernelized local-sensitivity constraint and the group sparsity constraint, the embedded structure information can be better explored, and significant performance improvement can be achieved. On the one hand, experiments on the ORL, AR, extended Yale B, and LFW data sets verify the superiority of our method. On the other hand, experiments on two unconstrained data sets, the LFW and the IJB-A, show that the utilization of sparsity can improve recognition performance, especially on the data sets with large pose variation.

Pose-invariant 3D face recognition using half face

Pose variations are still challenging problems in 3D face recognition because large pose variations will cause self-occlusion and result in missing data. In this paper, a new method for pose-invariant 3D face recognition is proposed to handle significant pose variations. For pose estimation and registration, a coarse-to-fine strategy is proposed to detect landmarks under large yaw variations. At the coarse search step, candidate landmarks are detected using HK curvature analysis and subdivided according to a facial geometrical structure-based classification strategy. At the fine search step, candidate landmarks are identified and labeled by comparing with a Facial Landmark Model. By using the half face matching, we perform the matching step with respect to frontal scans and side scans. Experiments carried out on the Bosphorus and UND/FRGC v2.0 databases show that our method has high accuracy and robustness to pose variations.

Supervised coordinate descent method with a 3D bilinear model for face alignment and tracking

AbstractFace alignment and tracking play important roles in facial performance capture. Existing data‐driven methods for monocular videos suffer from large variations of pose and expression. In this paper, we propose an efficient and robust method for this task by introducing a novel supervised coordinate descent method with 3D bilinear representation. Instead of learning the mapping between the whole parameters and image features directly with a cascaded regression framework in current methods, we learn individual sets of parameters mappings separately step by step by a coordinate descent mean. Because different parameters make different contributions to the displacement of facial landmarks, our method is more discriminative to current whole‐parameter cascaded regression methods. Benefiting from a 3D bilinear model learned from public databases, the proposed method can handle the head pose changes and extreme expressions out of plane better than other 2D‐based methods. We present the reliable result of face tracking under various head poses and facial expressions on challenging video sequences collected online. The experimental results show that our method outperforms state‐of‐art data‐driven methods.

Consistency-Constrained Nonnegative Coding for Tracking

A novel visual object tracking method based on consistency-constrained nonnegative coding (CNC) is proposed in this paper. For the purpose of computational efficiency, superpixels are first extracted from each observed video frame. And then CNC is performed based on those obtained superpixels, where the locality on manifold is preserved by enforcing the temporal and spatial smoothness. The coding result is achieved via an iterative update scheme, which is proved to converge. The proposed method enhances the coding stability and makes the tracker more robust for object tracking. The tracking performance has been evaluated based on ten challenging benchmark sequences involving drastic motion, partial or severe occlusions, large variation in pose, and illumination variation. The experimental results demonstrate the superior performance of our method in comparison with ten state-of-art trackers.

Locality-constrained linear coding based bi-layer model for multi-view facial expression recognition

Multi-view facial expression recognition is a challenging and active research area in computer vision. In this paper, we propose a simple yet effective method, called the locality-constrained linear coding based bi-layer (LLCBL) model, to learn discriminative representation for multi-view facial expression recognition. To address the issue of large pose variations, locality-constrained linear coding is adopted to construct an overall bag-of-features model, which is then used to extract overall features as well as estimate poses in the first layer. In the second layer, we establish one specific view-dependent model for each view, respectively. After the pose information of the facial image is known, we use the corresponding view-dependent model in the second layer to further extract features. By combining all the features in these two layers, we obtain a unified representation of the image. To evaluate the proposed approach, we conduct extensive experiments on both BU-3DFE and Multi-PIE databases. Experimental results show that our approach outperforms the state-of-the-art methods.

Deep Relative Tracking.

Most existing tracking methods are direct trackers, which directly exploit foreground or/and background information for object appearance modeling and decide whether an image patch is target object or not. As a result, these trackers cannot perform well when target appearance changes heavily and becomes different from its model. To deal with this issue, we propose a novel relative tracker, which can effectively exploit the relative relationship among image patches from both foreground and background for object appearance modeling. Different from direct trackers, the proposed relative tracker is robust to localize target object by use of the best image patch with the highest relative score to target appearance model. To model relative relationship among large-scale image patch pairs, we propose a novel and effective deep relative learning algorithm via Convolutional Neural Network. We test the proposed approach on challenging sequences involving heavy occlusion, drastic illumination changes, and large pose variations. Experimental results show that our method consistently outperforms state-of-the-art trackers due to the powerful capacity of the proposed deep relative model.

Feature Extraction and Fusion Using Deep Convolutional Neural Networks for Face Detection

This paper proposes a method that uses feature fusion to represent images better for face detection after feature extraction by deep convolutional neural network (DCNN). First, with Clarifai net and VGG Net-D (16 layers), we learn features from data, respectively; then we fuse features extracted from the two nets. To obtain more compact feature representation and mitigate computation complexity, we reduce the dimension of the fused features by PCA. Finally, we conduct face classification by SVM classifier for binary classification. In particular, we exploit offset max-pooling to extract features with sliding window densely, which leads to better matches of faces and detection windows; thus the detection result is more accurate. Experimental results show that our method can detect faces with severe occlusion and large variations in pose and scale. In particular, our method achieves 89.24% recall rate on FDDB and 97.19% average precision on AFW.

Robust discriminative regression for facial landmark localization under occlusion

Facial landmark localization or facial alignment is a crucial initial step in face analysis. The paper proposes a novel discriminative regression framework called Robust Discriminative Regression (RDR) for facial landmark localization. RDR framework consists of multiple partial feature regressors and a regression tree combination strategy. The proposed method copes with the partial facial landmarks invisible problem together with the optimization problem of multiple outputs combination. The RDR framework can be applied to both raw shape regression and model-based shape parameters regression. In model-based shape parameters regression we propose a two-level regression strategy, the first level is for rigid motion parameter regression and the second one is for non-rigid deformation parameter regression. Experiments on three widely used “face in-the-wild” databases (LFPW, COFW and IBUG) show that the proposed RDR outperforms other state-of-the-art facial landmark localization strategies in raw shape regression especially under partial occlusions or large pose variations. It also shows that the two-level regression strategy within RDR framework could achieve better performance than one-level parameters regression.

Multi-scale score level fusion of local descriptors for gender classification in the wild

The 2015 FRVT gender classification (GC) report evidences the problems that current approaches tackle in situations with large variations in pose, illumination, background and facial expression. The report suggests that both commercial and research solutions are hardly able to reach an accuracy over 90 % for The Images of Groups dataset, a proven scenario exhibiting unrestricted or in the wild conditions. In this paper, we focus on this challenging dataset, stepping forward in GC performance by observing: 1) recent literature results combining multiple local descriptors, and 2) the psychophysics evidences of the greater importance of the ocular and mouth areas to solve this task. We therefore make use of holistic and inner facial patches to extract features, that are later combined via a score level fusion strategy. The achieved results support the main information provided by the ocular and the mouth areas. Indeed, the combination of multiscale extracted features increases the overall accuracy to over 94 %, reducing notoriously the classification error if compared with tuned holistic and deep learning approaches.

Multi-Finger 3D Landmark Detection using Bi-Directional Hierarchical Regression

Over the past years, many application technologies related to Human Computer Interaction (HCI) have been developed for intelligent devices and wearable sensors. Moreover, they also have been received attention from both the research commu-nity and the industry. Almost of them make use of hand infor-mation such as hand gesture and posture. Consequently, the technical performance is highly depends on the information of humans’ hand and also, the number of products equipped with these technologies rapidly increases. Therefore, the accurate estimation of hand pose becomes a crucial issue to improve the performance of HCI applications. To detect landmarks, many algorithms have been develop-ing in this research field. Most of them are based on the pixel-based classification (1) and deformable part models (DPM) (2). However, they does not fit well in finger pose estimation since there are large pose variations due to camera viewpoints and various finger gestures. It is challenging to precisely detect the principal landmarks of fingers, as they flexibly deform with a large degree of freedom.In this paper, the regression based approach is used for esti-mating 3D multi-finger landmarks. Cascaded regression ap-proaches (3, 4) already used for hand pose estimation. Our ap-proach, however, concentrates on hierarchical structure of the detection strategy and initialization for better accuracy and stability. In our method, the bi-directional search structure is used for estimate landmarks with random forest regression. To

Point Cloud Based Relative Pose Estimation of a Satellite in Close Range.

Determination of the relative pose of satellites is essential in space rendezvous operations and on-orbit servicing missions. The key problems are the adoption of suitable sensor on board of a chaser and efficient techniques for pose estimation. This paper aims to estimate the pose of a target satellite in close range on the basis of its known model by using point cloud data generated by a flash LIDAR sensor. A novel model based pose estimation method is proposed; it includes a fast and reliable pose initial acquisition method based on global optimal searching by processing the dense point cloud data directly, and a pose tracking method based on Iterative Closest Point algorithm. Also, a simulation system is presented in this paper in order to evaluate the performance of the sensor and generate simulated sensor point cloud data. It also provides truth pose of the test target so that the pose estimation error can be quantified. To investigate the effectiveness of the proposed approach and achievable pose accuracy, numerical simulation experiments are performed; results demonstrate algorithm capability of operating with point cloud directly and large pose variations. Also, a field testing experiment is conducted and results show that the proposed method is effective.

Face pose normalization and simulation methods based on multi-view face alignment

In this paper, we propose a face pose normalization and simulation methods based on multi-view face alignment that can enhance the performance of the face recognition algorithm towards large pose variation. The proposed method includes two steps: 1) multi-view face alignment, 2) face pose normalization and simulation methods. Multi-view face alignment algorithm is inspired by the design idea of the Supervised Descent Method (SDM) which is considered the state-of-the-art in face alignment. The proposed method modified the algorithm to adapt multi-view problems by changing the histogram of gradient feature to projection of gradient feature in order to adapt large pose variance. In addition, the feature scale also can be adaptive adjusted towards different part of face, for example, eyes, mouth, eyebrows, etc. Based on the multi-view face alignment results, 2D face normalization and simulation methods are proposed. Experimental results over many images with obvious pose changes have shown our method can significantly normalize the multi-view pose face and improve the accuracy of the existing common face recognition method when faces of probe sets have large pose variation.

Robust Visual Tracking via Exclusive Context Modeling.

In this paper, we formulate particle filter-based object tracking as an exclusive sparse learning problem that exploits contextual information. To achieve this goal, we propose the context-aware exclusive sparse tracker (CEST) to model particle appearances as linear combinations of dictionary templates that are updated dynamically. Learning the representation of each particle is formulated as an exclusive sparse representation problem, where the overall dictionary is composed of multiple group dictionaries that can contain contextual information. With context, CEST is less prone to tracker drift. Interestingly, we show that the popular L1 tracker is a special case of our CEST formulation. The proposed learning problem is efficiently solved using an accelerated proximal gradient method that yields a sequence of closed form updates. To make the tracker much faster, we reduce the number of learning problems to be solved by using the dual problem to quickly and systematically rank and prune particles in each frame. We test our CEST tracker on challenging benchmark sequences that involve heavy occlusion, drastic illumination changes, and large pose variations. Experimental results show that CEST consistently outperforms state-of-the-art trackers.

Face recognition against pose variations using multi-resolution multiple colour fusion

In this paper we propose a new colour fusion approach called the multi-resolution MCF (MMCF) model. Unlike the recently proposed MCF and 2D_MCF using the colour images with same resolutions, this new model use the colour information of images with different resolutions with an aim to find one optimal combination of different colour combinations of images with different resolutions. More importantly, this new MMCF model can be used to face recognition against pose variations, especially when it is embedded with a deep learning network. Extensive experiments on seven different databases show the superiority of the proposed model over the MCF, 2D_MCF and RGB colour space, especially in the case of large pose variations, the corresponding improvements are significant. This research shed a light of future research on colour face recognition against pose variations.