Classification of Soft Biometric Traits When Matching Near-Infrared Long-Range Face Images Against Their Visible Counterparts

Abstract

In this chapter, we investigate the advantages and limitations of the heterogeneous problem of matching Near-Infrared (NIR) long-range, night time, face images against their visible counterparts. Image quality degradation can result due to a variety of factors including low illumination, variable standoff distance, and is responsible for performance degradation of conventional face recognition (FR) systems. In addition to intra-spectral matching (i.e. NIR vs. NIR face images), cross-spectral matching (i.e. matching NIR face images against their visible counterparts) is a challenging matching scenario that increases system complexity. In this work, we propose the usage of a set of FR algorithms when working with operational-based face matching scenarios, namely, where the face images used are collected by a night vision, long range (from 30 to 120 m), NIR-based face imaging system. First, we establish a system identification baseline using a set of commercial and academic face matchers. To improve baseline performance, we propose a scenario dependent convolutional neural network (CNN) to, first, categorize the face images of our challenging face dataset, in terms of gender, ethnicity, and facial hair. For each of the aforementioned generated categories, we apply our proposed algorithmic pipeline including, image restoration and a multi-feature based fusion scheme. Then, a set of FR algorithms are used before and after image restoration and data categorization. Based on the experimental results, we conclude that our proposed image restoration and fusion schemes, as well as the usage of demographic-based face categories, result in improved identification performance. For example, for the 30 m vs. 30 m NIR face matching scenario, the rank-1 identification rate is improved from 48% (all vs. all) using a commercial face matching system to 73% (all vs. all) and to 82% (if we use only the male with beards face data category). Experimental results suggest that our proposed methodological approach can improve system performance (i.e. efficiently identifying the subject of interest) on various cross-spectral face matching scenarios.