Pose Cues Research Articles

Driver distraction analysis using face pose cues

Vehicle driver distraction is one of the major reasons for road accidents. Involvement with a co-passenger, use of in-vehicle devices or phone leads to a situation where the driver head pose varies and the eye is off the road. A low cost early warning system should reduce the distracted driving instances, thus making our roads safer. Face pose information forms an important cue to determine driver distraction. The main objective of this work is to analyse the distractions of the driver based on his/her face pose cues. A straight pose or slight variation would indicate a non-distracted driver, while a large pose variation from the center would indicate a high probability for a distracted driver. Face pose database of vehicle drivers is developed and is bench-marked. A clustered two layer approach on Gabor features is proposed. A five layer convolutional network with three fully connected layers is also used to bench-mark the data. The proposed clustered two-layer approach with Gabor features and SVM classifier provides better results in driver distraction analysis when compared to the deep learning approach and other manifold approaches. The improved accuracy could be attributed to the improved modeling of manifold in our approach, better class discrimination of the Gabor features together with better classification provided by the SVM classifier.

PA-Net: Learning local features using by pose attention for short-term person re-identification

Person re-identification (Re-ID) is an important but challenging task in video surveillance applications. In Re-ID tasks, pose is an extremely useful cue to identify a person, even from the back view. Therefore, pose-detection models may learn the features that are beneficial to the Re-ID task and improve the Re-ID performance by fusing the feature maps into the Re-ID model. Two key problems in integrating the pose cues are addressed in this study. One is how to reduce the noise caused by cross-domain datasets. The other is how to fuse the feature maps to better utilize high-level semantic pose cues. To address these two key problems, we first propose PA-Net by combining the pose attention stream and the global attention stream, where the global attention stream distinguishes persons with different global appearances, and the pose attention stream distinguishes persons with similar global appearance but different poses. Then, we present a pose attention stream that learns local features to reduce the noise in the pose cues caused by the cross-domain datasets and provide more semantic information for the Re-ID task. The effects of the proposed pose attention are demonstrated in an ablation study, and comparative experiments show that PA-Net achieves state-of-the-art performance.

Automatic Registration Between Dental Cone-Beam CT and Scanned Surface via Deep Pose Regression Neural Networks and Clustered Similarities.

Computerized registration between maxillofacial cone-beam computed tomography (CT) images and a scanned dental model is an essential prerequisite for surgical planning for dental implants or orthognathic surgery. We propose a novel method that performs fully automatic registration between a cone-beam CT image and an optically scanned model. To build a robust and automatic initial registration method, deep pose regression neural networks are applied in a reduced domain (i.e., two-dimensional image). Subsequently, fine registration is performed using optimal clusters. A majority voting system achieves globally optimal transformations while each cluster attempts to optimize local transformation parameters. The coherency of clusters determines their candidacy for the optimal cluster set. The outlying regions in the iso-surface are effectively removed based on the consensus among the optimal clusters. The accuracy of registration is evaluated based on the Euclidean distance of 10 landmarks on a scanned model, which have been annotated by experts in the field. The experiments show that the registration accuracy of the proposed method, measured based on the landmark distance, outperforms the best performing existing method by 33.09%. In addition to achieving high accuracy, our proposed method neither requires human interactions nor priors (e.g., iso-surface extraction). The primary significance of our study is twofold: 1) the employment of lightweight neural networks, which indicates the applicability of neural networks in extracting pose cues that can be easily obtained and 2) the introduction of an optimal cluster-based registration method that can avoid metal artifacts during the matching procedures.

PoSeg: Pose-Aware Refinement Network for Human Instance Segmentation

Human instance segmentation is a core problem for human-centric scene understanding and segmenting human instances poses a unique challenge to vision systems due to large intra-class variations in both appearance and shape, and complicated occlusion patterns. In this paper, we propose a new pose-aware human instance segmentation method. Compared to the previous pose-aware methods which first predict bottom-up poses and then estimate instance segmentation on top of predicted poses, our method integrates both top-down and bottom-up cues for an instance: it adopts detection results as human proposals and jointly estimates human pose and instance segmentation for each proposal. We develop a modular recurrent deep network that utilizes pose estimation to refine instance segmentation in an iterative manner. Our refinement modules exploit pose cues in two levels: as a coarse shape prior and local part attention. We evaluate our approach on two public multi-person benchmarks: OCHuman dataset and COCOPersons dataset. The proposed method surpasses the state-of-the-art methods on OCHuman dataset by 3.0 mAP and on COCOPersons by 6.4 mAP, demonstrating the effectiveness of our approach.

Pigeons and humans use action and pose information to categorize complex human behaviors

The biological mechanisms used to categorize and recognize behaviors are poorly understood in both human and non-human animals. Using animated digital models, we have recently shown that pigeons can categorize different locomotive animal gaits and types of complex human behaviors. In the current experiments, pigeons (go/no-go task) and humans (choice task) both learned to conditionally categorize two categories of human behaviors that did not repeat and were comprised of the coordinated motions of multiple limbs. These “martial arts” and “Indian dance” action sequences were depicted by a digital human model. Depending upon whether the model was in motion or not, each species was required to engage in different and opposing responses to the two behavioral categories. Both species learned to conditionally and correctly act on this dynamic and static behavioral information, indicating that both species use a combination of static pose cues that are available from stimulus onset in addition to less rapidly available action information in order to successfully discriminate between the behaviors. Human participants additionally demonstrated a bias towards the dynamic information in the display when re-learning the task. Theories that rely on generalized, non-specific visual mechanisms involving channels for motion and static cues offer a parsimonious account of how humans and pigeons recognize and categorize behaviors within and across species.

Gesture3D

Artists routinely use gesture drawings to communicate ideated character poses for storyboarding and other digital media. During subsequent posing of the 3D character models, they use these drawing as a reference, and perform the posing itself using 3D interfaces which require time and expert 3D knowledge to operate. We propose the first method for automatically posing 3D characters directly using gesture drawings as an input, sidestepping the manual 3D posing step. We observe that artists are skilled at quickly and effectively conveying poses using such drawings, and design them to facilitate a single perceptually consistent pose interpretation by viewers. Our algorithm leverages perceptual cues to parse the drawings and recover the artist-intended poses. It takes as input a vector-format rough gesture drawing and a rigged 3D character model, and plausibly poses the character to conform to the depicted pose. No other input is required. Our contribution is two-fold: we first analyze and formulate the pose cues encoded in gesture drawings; we then employ these cues to compute a plausible image space projection of the conveyed pose and to imbue it with depth. Our framework is designed to robustly overcome errors and inaccuracies frequent in typical gesture drawings. We exhibit a wide variety of character models posed by our method created from gesture drawings of complex poses, including poses with occlusions and foreshortening. We validate our approach via result comparisons to artist-posed models generated from the same reference drawings, via studies that confirm that our results agree with viewer perception, and via comparison to algorithmic alternatives.

Local Surface Geometric Feature for 3D human action recognition

This paper presents a novel Local Surface Geometric Feature (LSGF) for human action recognition from video sequences captured by a depth camera. The LSGF is extracted from each skeleton joint in point cloud space to capture the static appearance and pose cues, which includes joint position, normal, and local curvature. A temporal pyramid of covariance matrix is exploited to model both pairwise relations of features instead of features themselves and the temporal evolution. Finally, Fisher vector encoding is imported as a global representation for a video sequence and SVM classifier is used for classification. In the extensive experiments, we achieve classification results superior to most of previous published results on three public benchmark datasets, i.e., MSR-Action3D, MSR DailyActivity3D, and UTKinect Action.

Pose-Guided Human Parsing by an AND/OR Graph Using Pose-Context Features

Parsing human into semantic parts is crucial to human-centric analysis. In this paper, we propose a human parsing pipeline that uses pose cues, e.g., estimates of human joint locations, to provide pose-guided segment proposals for semantic parts. These segment proposals are ranked using standard appearance cues, deep-learned semantic feature, and a novel pose feature called pose-context. Then these proposals are selected and assembled using an And-Or graph to output a parse of the person. The And-Or graph is able to deal with large human appearance variability due to pose, choice of clothing, etc. We evaluate our approach on the popular Penn-Fudan pedestrian parsing dataset, showing that it significantly outperforms the state of the art, and perform diagnostics to demonstrate the effectiveness of different stages of our pipeline.