Occlusion Detection and Localization from Kinect Depth Images

Abstract

Faces captured in a real-world scenario may suffer from large variations in shape and occlusions due to difference in illumination, variation in pose and orientation of a facial image. Automated face recognition or security reinforcement by surveillance techniques would be useless if the faces are occluded. Therefore, face occlusion detection has become very important not only for effective face recognition but also to prevent security threats. In this paper, for the very first time an occlusion detection method is proposed based on the depth information provided by Kinect RGB-D cameras. Uniform Local Binary Pattern (LBP) is used to effectively extract the features from the depth images and SVM binary classifier is then applied to identify the front face and the occluded face. For localizing occluded regions in the face image, a threshold based approach is proposed to identify the areas close to the camera. In the depth images, an object close to the camera has a higher pixel intensity than the object further from the camera. Thus, we assume that occluded regions have lower distance from the camera, i.e. higher intensity values. Based on this hypothesis, we extract the connected component with highest energy values as the potential occluded region from the depth image. The boundary of the detected occluded region is then corrected using the reference front face image. The occlusion detection and localization method have been evaluated on EUROKOM Kinect face database containing different types of occluded and unoccluded faces with neutral expressions. Experimental results show that the proposed method provides an average detection rate of 98.50% for front and occluded face images. We have also compared our proposed method with existing methods that use faces acquired using 3D scanners for occlusion detection.