Non-cooperative and Occluded Person Identification Using Periocular Region with Visible, Infra-Red, and Hyperspectral Imaging

Abstract

The performance of automatic person identification based on visual appearance significantly suffers under occlusions in many real life situations. These occlusions may be unintentional due to the use of different items such as head gear, headphone, head scarf, or may also be caused by the style of clothing or just hair style. Intentional facial occlusions occur when a particular person try to hide his identity by hiding his face and appearance from the security cameras. In many incidents captured by surveillance videos, it has been observed that the offenders have covered their appearance and faces from the camera, leaving only the small region around the eyes known as “periocular region.” It is because the periocular region cannot be covered to maintain proper vision. In this chapter we present an extensive study on periocular region based person identification using videos in the visible spectrum, near IR range, and also by using the hyperspectral image cubes in a relatively wider bandwidth. While most of the existing techniques for periocular recognition from videos have handpicked a single best frame from videos, we formulate periocular region based person identification in video as an image-set classification problem. For thorough analysis, we perform experiments on periocular regions extracted automatically from RGB videos, NIR videos, and hyperspectral image cubes. Each image-set is represented by four heterogeneous feature types and classified with six state-of-the-art image-set classification algorithms. We will discuss in detail our novel two stage inverse Error Weighted Fusion algorithm for feature and classifier score fusion. We observe that the proposed two stage fusion is superior to single stage fusion. Comprehensive experimental results are presented on four publicly available datasets including Multiple Biometric Grand Challenge (MBGC) NIR, MBGC visible spectrum dataset both by NIST, Carnegie Mellon University (CMU) Hyperspectral face database (Tech. Report CMURI-TR-02-25), and University of Beira Interior Periocular (UBIPr) dataset. In these experiments excellent recognition on all of the four datasets has been observed. These results are significantly better than the result of most of the existing state-of-the-art methods on the same datasets and under similar experimental setup. In addition to these improvements, we demonstrate the feasibility of image-set based periocular biometrics for real world applications. Deployment of security systems with periocular region based person identification algorithm will reduce the vulnerability of security systems to be hacked by non-cooperative individuals.