Real-time Head Pose Estimation Research Articles

A Novel Zernike Moment-Based Real-Time Head Pose and Gaze Estimation Framework for Accuracy-Sensitive Applications.

A real-time head pose and gaze estimation (HPGE) algorithm has excellent potential for technological advancements either in human-machine or human-robot interactions. For example, in high-accuracy advent applications such as Driver's Assistance System (DAS), HPGE plays a crucial role in omitting accidents and road hazards. In this paper, the authors propose a new hybrid framework for improved estimation by combining both the appearance and geometric-based conventional methods to extract local and global features. Therefore, the Zernike moments algorithm has been prominent in extracting rotation, scale, and illumination invariant features. Later, conventional discriminant algorithms were used to classify the head poses and gaze direction. Furthermore, the experiments were performed on standard datasets and real-time images to analyze the accuracy of the proposed algorithm. As a result, the proposed framework has immediately estimated the range of direction changes under different illumination conditions. We obtained an accuracy of ~85%; the average response time was 21.52 and 7.483 ms for estimating head poses and gaze, respectively, independent of illumination, background, and occlusion. The proposed method is promising for future developments of a robust system that is invariant even to blurring conditions and thus reaching much more significant performance enhancement.

Real-time head pose estimation using multi-task deep neural network

Driver inattention is one of the main causes of traffic accidents. To avoid such accidents, advanced driver assistance system that passively monitors the driver’s activities is needed. In this paper, we present a novel method to estimate a head pose from a monocular camera. The proposed algorithm is based on multi-task learning deep neural network that uses a small grayscale image. The network jointly detects multi-view faces and estimates head pose even under poor environment conditions such as illumination change, vibration, large pose change, and occlusion. We also propose a multi-task learning method that does not bias on a specific task with different datasets. Moreover, in order to fertilize training dataset, we establish and release the RCVFace dataset that has accurate head poses. The proposed framework outperforms state-of-the-art approaches quantitatively and qualitatively with an average head pose mean error of less than 4° in real-time. The algorithm applies to driver monitoring system that is crucial for driver safety.

NUI framework based on real-time head pose estimation and hand gesture recognition

The natural user interface (NUI) is used for the natural motion interface without using device or tool such as mice, keyboards, pens and markers. In this paper, we develop natural user interface framework based on two recognition module. First module is real-time head pose estimation module using random forests and second module is hand gesture recognition module, named Hand gesture Key Emulation Toolkit (HandGKET). Using the head pose estimation module, we can know where the user is looking and what the user’s focus of attention is. Moreover, using the hand gesture recognition module, we can also control the computer using the user’s hand gesture without mouse and keyboard. In proposed framework, the user’s head direction and hand gesture are mapped into mouse and keyboard event, respectively.

Enhanced real-time head pose estimation system for mobile device

This article proposes a real-time Head Pose Estimation (HPE) technique designed to be used in mobile devices. The method enables the interaction between the user and mobile devices using the device's inbuilt camera. The proposed technique is composed

KinectFusion for Faces: Real-Time 3D Face Tracking and Modeling Using a Kinect Camera for a Markerless AR System

In this paper we present an extension to the KinectFusion algorithm that allows a robust real-time face tracking and modeling using the Microsoft’s Kinect sensor. This is achieved changing two steps of the original algorithm: pre-processing and tracking. In the former, we use a real-time face detection algorithm to segment the face from the rest of the image. In the latter, we use a real-time head pose estimation to give a new initial guess to the Iterative Closest Point (ICP) algorithm when it fails and an algorithm to solve occlusion.Our approach is evaluated in a markerless augmented reality (MAR) system. We show that this approach can reconstruct faces and handle more face pose changes and variations than the original KinectFusion’s tracking algorithm. In addition, we show that the realism of the system is enhanced as we solve the occlusion problem efficiently at shader level.

Dynamic random regression forests for real-time head pose estimation

For real-time evaluation of the position and orientation of the human head using depth image, we propose a novel algorithm, the dynamic random regression forests (DRRF), which enhances the conventional random forests (RF) in four aspects. Firstly, the DRRF employs the boosting strategy for data induction to upgrade the learning quality; secondly, the key parameters are optimized in a dynamic manner in order to train the DRRF classifier efficiently; thirdly, a stem operator is integrated into the conventional tree-shaped classifier to increase the possibility of optimum data split; fourthly, a weighted voting scheme utilizes the learning knowledge to determine the regression result more efficiently and accurately. Comparative experiments verify the advantages of the aforementioned four improvement schemes, and demonstrate the DRRF's accuracy and robustness against partial occlusion and the variations of head pose, illumination, and facial expression.

Random Forests for Real Time 3D Face Analysis

We present a random forest-based framework for real time head pose estimation from depth images and extend it to localize a set of facial features in 3D. Our algorithm takes a voting approach, where each patch extracted from the depth image can directly cast a vote for the head pose or each of the facial features. Our system proves capable of handling large rotations, partial occlusions, and the noisy depth data acquired using commercial sensors. Moreover, the algorithm works on each frame independently and achieves real time performance without resorting to parallel computations on a GPU. We present extensive experiments on publicly available, challenging datasets and present a new annotated head pose database recorded using a Microsoft Kinect.