Abstract
In this paper we present an extension to the KinectFusion algorithm that allows a robust real-time face tracking and modeling using the Microsoft’s Kinect sensor. This is achieved changing two steps of the original algorithm: pre-processing and tracking. In the former, we use a real-time face detection algorithm to segment the face from the rest of the image. In the latter, we use a real-time head pose estimation to give a new initial guess to the Iterative Closest Point (ICP) algorithm when it fails and an algorithm to solve occlusion.Our approach is evaluated in a markerless augmented reality (MAR) system. We show that this approach can reconstruct faces and handle more face pose changes and variations than the original KinectFusion’s tracking algorithm. In addition, we show that the realism of the system is enhanced as we solve the occlusion problem efficiently at shader level.
Highlights
Augmented reality (AR) is a technology in which a user’s view of a real scene is augmented with additional virtual information
RELATED WORK Surface reconstruction, face modeling, markerless AR and head pose estimation have been driven by different approaches, as we can see in the subsections
WORK We have presented the KinectFusion for Faces: an approach for real-time face tracking and modeling using a Kinect camera for a markerless AR system
Summary
Augmented reality (AR) is a technology in which a user’s view of a real scene is augmented with additional virtual information. In some AR systems, the user turns his head in front of a camera and the head is augmented with a virtual object In this case, is desirable an algorithm able to track the person’s head with enough accuracy and in real-time. We present an approach for robust real-time face tracking and modeling using the Microsoft’s Kinect sensor for a markerless augmented reality (MAR) system. Our approach adapts the KinectFusion to the face modeling and extends its tracking using the head pose estimation We show that this approach can reconstruct faces and handle more face pose changes and variations than the original KinectFusion’s tracking algorithm.
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
