Robust joint representation with triple local feature for face recognition with single sample per person

Abstract

Face recognition (FR) with a single training sample per person (SSPP) is a representative small-sample-size classification problem and occurs in many practical scenarios such as law enforcement, surveillance, identity card, e-passport, etc. By using intra-class variations extracted from an additional training set (i.e., a set excluding gallery subjects) as a generic intra-class variation dictionary, sparse representation based classification (SRC) has been extended to FR with SSPP. However, for FR with SSPP, how to achieve high robustness to gross facial variations (e.g., complex facial lighting, expression and pose variations and various outliers of corruption, occlusion and disguise) is still an open issue. In this paper, we propose a novel model, named robust joint representation with triple local feature (RJR-TLF), to address this issue from the viewpoints of feature extraction and classifier design. In feature extraction, we design robust triple local features, i.e., Gabor facial features with multiple scales and multiple orientations extracted in different facial local regions (e.g., local patches centered around dense regularly sampled points and detected particular points including nose tip, eye centers, etc.), to naturally encode the local scale, local orientation and local space information of a face image. For face images, the densely and regularly sampled facial regions can provide a comprehensive description; the sparsely and particularly detected facial regions can exploit a discriminative description because they cover the most informative facial regions and can be detected robustly. In classifier design, we propose a robust joint representation framework to exploit the distinctiveness and similarity of different local information by requiring triple local features from the same type of Gabor feature (i.e., with the same scale and orientation) to have similar representation coefficients. With the coefficient-similarity constraint and the robust representation fidelity term representing the query image on the single-sample gallery set and the generic intra-class variation dictionary, the local features with large-representation residuals actually indicate corrupted regions with gross facial variations and will be assigned low weights adaptively to reduce their effects on the representation and classification, which further strengthens the robustness of RJR-TLF. The proposed RJR-TLF is evaluated extensively on popular databases, including the AR, the large-scale CMU Multi-PIE, and the LFW databases. Experimental results demonstrate that RJR-TLF is much more robust to various facial variations than the recent FR with SSPP methods.