Hyperspectral Face Recognition Research Articles

HyperFace: A Deep Fusion Model for Hyperspectral Face Recognition.

Face recognition has been well studied under visible light and infrared (IR) in both intra-spectral and cross-spectral cases. However, how to fuse different light bands for face recognition, i.e., hyperspectral face recognition, is still an open research problem, which has the advantages of richer information retention and all-weather functionality over single-band face recognition. Thus, in this research, we revisit the hyperspectral recognition problem and provide a deep learning-based approach. A new fusion model (named HyperFace) is proposed to address this problem. The proposed model features a pre-fusion scheme, a Siamese encoder with bi-scope residual dense learning, a feedback-style decoder, and a recognition-oriented composite loss function. Experiments demonstrate that our method yields a much higher recognition rate than face recognition using only visible light or IR data. Moreover, our fusion model is shown to be superior to other general-purpose image fusion methods that are either traditional or deep learning-based, including state-of-the-art methods, in terms of both image quality and recognition performance.

Hyperspectral Face Recognition with Adaptive and Parallel SVMs in Partially Hidden Face Scenarios.

Hyperspectral imaging opens up new opportunities for masked face recognition via discrimination of the spectral information obtained by hyperspectral sensors. In this work, we present a novel algorithm to extract facial spectral-features from different regions of interests by performing computer vision techniques over the hyperspectral images, particularly Histogram of Oriented Gradients. We have applied this algorithm over the UWA-HSFD dataset to extract the facial spectral-features and then a set of parallel Support Vector Machines with custom kernels, based on the cosine similarity and Euclidean distance, have been trained on fly to classify unknown subjects/faces according to the distance of the visible facial spectral-features, i.e., the regions that are not concealed by a face mask or scarf. The results draw up an optimal trade-off between recognition accuracy and compression ratio in accordance with the facial regions that are not occluded.

Hyperspectral face recognition with histogram of oriented gradient features and collaborative representation-based classifier

Hyperspectral face recognition with histogram of oriented gradient features and collaborative representation-based classifier

Scale fusion light CNN for hyperspectral face recognition with knowledge distillation and attention mechanism

Scale fusion light CNN for hyperspectral face recognition with knowledge distillation and attention mechanism

Hyperspectral-cube-based mobile face recognition: A comprehensive review

With the hyperspectral sensor technology evolving and becoming more cost-effective, hyperspectral imaging offers new opportunities for robust face recognition. Hyperspectral face cubes contain much more spectral information than face images from common RGB color cameras. Hyperspectral face recognition is robust to the impacts, such as illumination, pose, occlusion, and spoofing, which can heavily avoid the limitations of the visible-image-based face recognition.In this paper, we summarize the spectrum properties of hyperspectral face cubes and survey the hyperspectral face recognition methods in the literature. We categorize them into major groups for better understanding. We overview the existing hyperspectral face datasets, and establish our own dataset. We also discuss efficient neural networks used for mobile face recognition and conduct experiments on mobile hyperspectral face recognition. Results show that under harsh conditions like large illumination changing and pose variation, hyperspectral-cube-based methods have higher recognition accuracy than visible-image-based methods. Finally, we deliver insightful discussions and prospects for future works on mobile hyperspectral face recognition.

Hyperspectral face recognition with a spatial information fusion for local dynamic texture patterns and collaborative representation classifier

Hyperspectral face recognition provides improved classification rates due to its abundant information in the face cubes of every subject in hyperspectral face databases. However, while offering excellent opportunities, it also brings new challenges, such as low signal-to-noise ratio, interband misalignment, and high data dimensionality. Based on these ad hoc problems, literature has already proposed some optimisation methods including dimensionality reduction, image denoising, and alignment to perform face recognition, yet lacking comprehensive evaluation. This paper proposes a novel hyperspectral face recognition algorithm that is based on spatial information fusion for feature extraction (histogram of local dynamic texture patterns) and collaborative representation classifier for classification. Meanwhile, the algorithm is applied to three popular hyperspectral face databases, Carnegie Mellon University (CMU)-hyperspectral face database (HSFD), University of Western Australia (UWA)-HSFD, and Hong Kong Polytechnic University (PolyU)-HSFD databases. Experimental results demonstrate that CMU-HSFD and UWA-HSFD databases achieve very competitive classification results. PolyU-HSFD database also achieves rather good classification rates. The best recognition results are 98.5% ± 0.95, 96.6% ± 0.98, and 94.0% ± 2.86 for CMU-HSFD, UWA-HSFD and PolyU-HSFD, respectively. It demonstrates experimentally that this algorithm can be used to recognise faces. Moreover, we compared eight existing state-of-the-art face recognition techniques with our proposed method in performing hyperspectral face recognition. In this research, we formulate hyperspectral face recognition as an image-set classification problem and evaluate the performances compared with other kinds of algorithms. Comparisons with the eight existing hyperspectral face recognition techniques on three standard datasets show that the proposed algorithm outperforms most other state-of-the-art algorithms, indicating that it is a promising approach for hyperspectral face recognition.

Regularization and attention feature distillation base on light CNN for Hyperspectral face recognition

The hyperspectral imaging, capturing discriminative information across a series of spectrum bands, leads to building a robust face recognition system. Motivated by the success of deep convolutional network and transferring learning, this paper proposed an end-to-end hyperspectral face recognition model based on a light Convolutional Neural Network (CNN) and transfer learning. To boost the performance of hyperspectral face recognition, the Max-Feature-Map (MFM) activation function and fine-tuning (structure regularization(L2SP) or attention feature distillation (AFD)) are introduced to optimize the deep network, which will learn the fine feature representation across different bands. Especially, this method incorporates regularization and AFD cooperation into the transfer learning strategy on the visible face data. By feeding back hyperspectral images to the pretrained light CNN network, we can design an end-to end model that can leverage the generalization ability for hyperspectral face images. Finally, the entire model is trained and verified on the PolyU-HSFD, CMU, and UWA hyperspectral face datasets using the associated standard evaluation protocols. Experimental results demonstrate that the improved Light CNN network can get good representations of hyperspectral face features and the joint training with a combination of L2SP and AFD exhibits better recognition performance than the state-of-the-art methods based on other deep networks.

Hyperspectral face recognition based on sparse spectral attention deep neural networks.

Inspired by the robust capability and outstanding performance of convolutional neural networks (CNN) in image classification tasks, CNN-based hyperspectral face recognition methods are worthy of further exploration. However, hyperspectral imaging poses new challenges including high data dimensionality and interference between bands on spectral dimension. High data dimensionality can result in high computational costs. Moreover, not all bands are equally informative and discriminative. The usage of a useless spectral band may even introduce noises and weaken the performance. For the sake of solving those problems, we proposed a novel CNN framework, which adopted a channel-wise attention mechanism and Lasso algorithm to select the optimal spectral bands. The framework is termed as the sparse spectral channel-wise attention-based network (SSCANet) where the SSCA-block focuses on the inter-band channel relationship. Different from other methods which usually select the useful bands manually or in a greedy fashion, SSCA-block can adaptively recalibrate spectral bands by selectively emphasizing informative bands and suppressing less useful ones. Especially, a Lasso constraint strategy can zero out the bands during the training of the network, which can boost the training process by making the weights of bands sparser. Finally, we evaluate the performance of the proposed method in comparison of other state-of-the-art hyperspectral face recognition algorithms on three public datasets HK-PolyU, CMU, and UWA. The experimental results demonstrate that SSCANet based method outperforms the state-of-the-art methods for face recognition on the benchmark.

Combining pixel selection with covariance similarity approach in hyperspectral face recognition based on convolution neural network

A Hyperspectral camera provides discriminating features for capturing human faces that cannot be obtained by any other imaging technique. Nevertheless, it has new issues comprising curse of dimensionality, physical parameter retrieval, fast computing and inter band misalignment. As a result of, the literature of Hyperspectral Face Recognition is more scanty and confined to improvised dimensionality reduction and minimization of wide-ranging bands, and thus the objective can be obtained by the use of the Convolution Neural Network (ConvNet). Since ConvNet is of great attention in recent times, it can offer outstanding performance in face recognition systems, where the quantity of training data is amply large. We propose a Hyperspectral Face Recognition system using Firefly algorithm for band fusion and the Convolution Neural Network for classification. In addition to this, the present work is extended 11 exiting face recognition methods to perform Hyperspectral Face Recognition task. Thus the work has been framed as Hyperspectral Face Recognition problem to an image-set classification problem and assessment of the performance has been done on six state-of-the-art image-set problem techniques, and similarly it was examined on five state-of-the-art RGB and gray scale face recognition system, subsequently applied improved Firefly band selection algorithm on Hyperspectral Images to get appropriate band. Assessment with the eleven extended and five existing HSI Face Recognition system on two benchmark datasets (CMU-HSFD & UWA-HSFD) demonstrates that the proposed system overtakes all by a noteworthy margin. Lastly, we execute the band selection demonstration to get the novelty for most informative bands in Visible Near Infrared (VNIR).

Development of spectral signatures and classification using hyperspectral face recognition

Hyperspectral facial dataset represent innovative statistics, as compared to the traditional images and control the statistics in the subbands of Electromagnetic Spectrum (EMS) over a continuous range and produce the spectral libraries of all facial images. The research is accomplished on Carnegie Mellon University (CMU) Hyperspectral Face Datasets (HFDS) within the spectral series of 610nm to 1100 nm (VIR-NIR), having 50 spectral bands using ENVI 4.8 Software. Spectral Libraries are developed for different face attributes to identify faces for a period of time, even in the occurrence of changes in facial appearance. Supervised classification is performed by using classification techniques Spectral Angle Mapper (SAM). Experiments are accompanied to show the simplicity of the algorithm to classify and to develop spectral curves for hyperspectral face images.

Hyperspectral face recognition with minimum noise fraction, histogram of oriented gradient features and collaborative representation-based classifier

Hyperspectral face recognition with minimum noise fraction, histogram of oriented gradient features and collaborative representation-based classifier

Hierarchical Clustering Based Band Selection Algorithm for Hyperspectral Face Recognition

Hyperspectral face recognition is a small sample size problem, where usually less than four hyperspectral cubes are available as training data. At the same time, hyperspectral face image acquires grayscale images over a series of continuous spectra which usually contain large redundant information or noise, especially in the near infrared spectrum bands. Therefore, dimensionality reduction and feature extraction are important tasks on this problem. This paper proposes a hierarchical clustering-based spectrum band selection method, which mitigates the influence of noise and extracts features from each spectra band by using the Gabor filter and the histograms of oriented gradients algorithm, In addition, the fusion of Hog and Gabor features was embedded into the nearest neighborhood-based classifier for performance comparison. The experimental results show that the proposed algorithm is time effective and provides robust performance.

Hyperspectral Face Recognition with Patch-Based Low Rank Tensor Decomposition and PFFT Algorithm

Hyperspectral imaging technology with sufficiently discriminative spectral and spatial information brings new opportunities for robust facial image recognition. However, hyperspectral imaging poses several challenges including a low signal-to-noise ratio (SNR), intra-person misalignment of wavelength bands, and a high data dimensionality. Many studies have proven that both global and local facial features play an important role in face recognition. This research proposed a novel local features extraction algorithm for hyperspectral facial images using local patch based low-rank tensor decomposition that also preserves the neighborhood relationship and spectral dimension information. Additionally, global contour features were extracted using the polar discrete fast Fourier transform (PFFT) algorithm, which addresses many challenges relevant to human face recognition such as illumination, expression, asymmetrical (orientation), and aging changes. Furthermore, an ensemble classifier was developed by combining the obtained local and global features. The proposed method was evaluated by using the Poly-U Database and was compared with other existing hyperspectral face recognition algorithms. The illustrative numerical results demonstrate that the proposed algorithm is competitive with the best CRC_RLS and PLS methods.

3D discrete wavelet transform‐based feature extraction for hyperspectral face recognition

Facial hyperspectral image analysis has become a popular topic since it provides additional spectral information on subjects unlike the 2D face imagery which only has spatial information, hence it has an opportunity to improve face recognition accuracy. Three new methods for feature extraction for facial hyperspectral image classification are proposed. The methods employ a three-dimensional discrete wavelet transform (3D-DWT) to extract features from facial hyperspectral images. One of the advantages of 3D-DWT for feature extraction in hyperspectral images is that the horizontal, vertical and spectral information are processed in parallel. The most important characteristic of 3D-DWT is decomposing hyperspectral images into a set of spatio-spectral frequency subbands. The study proposes three methods using 3D-DWT for feature extraction: 3D-subband energy, 3D-subband overlapping cube and 3D-global energy. The k-NN and collaborative representation-based classifier (CRC) are used to process extracted feature vector datasets, where classification accuracies are evaluated by four test scenarios. The results under different test scenarios revealed that accuracy of proposed 3D-DWT methods is superior to alternative methods using spatio-spectral classification.

Hyperspectral face recognition via feature extraction and CRC‐based classifier

Hyperspectral face recognition provides improved classification rates due to its abundant information in the face cubes of every subject in hyperspectral face databases. However, it is less popular in face recognition due to its difficulty in data acquisition, low signal-to-noise ratio, and high dimensionality. The authors compare five existing descriptors that are frequently used in 2D face recognition, and use collaborative representation classifier (CRC) with two voting techniques for hyperspectral face recognition. Experimental results demonstrate that, for PolyU-HSFD database, Gabor filter bank-based features are very robust to both Gaussian white noise and shot noise, and it achieves very competitive classification results. For CMU-HSFD database, when the noise level is low, histogram of oriented gradients (HOG) yields good classification results. In addition, when the noise level is high, raw facial images without feature extraction perform very well in term of correct classification rate. The local binary pattern and HOG descriptor are very sensitive to noise even though they achieve rather good classification rates if the facial images contain no noise. The best recognition result for the PolyU-HSFD is 96.4% ± 2.3 and for the CMU-HSFD is 98.0% ± 0.7.

On the usefulness of hyperspectral imaging for face recognition

Hyperspectral cameras provide additional information in terms of multiple sampling of the visible spectrum, holding information that could be potentially useful for biometric applications. This paper investigates whether the performance of hyperspectral face recognition algorithms can be improved by considering single and multiple one-dimensional (1-D) projections of the whole spectral data along the spectral dimension. Three different projections are investigated and found by optimization: single-spectral band selection, nonnegative spectral band combination, and unbounded spectral band combination. Since 1-D projections can be performed directly on the imaging device with color filters, projections are also restricted to be physically plausible. The experiments are performed on a standard hyperspectral dataset and the obtained results outperform eight existing hyperspectral face recognition algorithms.

Hyperspectral face recognition using improved inter-channel alignment based on qualitative prediction models.

A fundamental limitation of hyperspectral imaging is the inter-band misalignment correlated with subject motion during data acquisition. One way of resolving this problem is to assess the alignment quality of hyperspectral image cubes derived from the state-of-the-art alignment methods. In this paper, we present an automatic selection framework for the optimal alignment method to improve the performance of face recognition. Specifically, we develop two qualitative prediction models based on: 1) a principal curvature map for evaluating the similarity index between sequential target bands and a reference band in the hyperspectral image cube as a full-reference metric; and 2) the cumulative probability of target colors in the HSV color space for evaluating the alignment index of a single sRGB image rendered using all of the bands of the hyperspectral image cube as a no-reference metric. We verify the efficacy of the proposed metrics on a new large-scale database, demonstrating a higher prediction accuracy in determining improved alignment compared to two full-reference and five no-reference image quality metrics. We also validate the ability of the proposed framework to improve hyperspectral face recognition.

Hyperspectral face recognition with log‐polar Fourier features and collaborative representation based voting classifiers

Hyperspectral imagery analysis has become a popular topic for improving face recognition accuracy. Nevertheless, it encounters difficulty in data acquisition, low signal-to-noise ratio, and high dimensionality. As a result, there exists a need to develop better algorithms in order to achieve higher classification rates. In this study, the authors propose a new method for hyperspectral face recognition with very competitive experimental results. Since there is a significant amount of noise in every spectral band, they reduce noise adaptively from each spectral band by using any image denoising method, e.g. block matching and 3D filtering. They then crop each face according to its eye coordinates so that translation invariance can be achieved. They conduct log-polar transform to each cropped face image and extract 2D Fourier spectrum from them. In this way, the extracted features are approximately invariant to translation, rotation, and scaling. They use the collaborative representation-based classifier with voting for hyperspectral face recognition. They perform some experiments to test the authors’ new method for hyperspectral face recognition with very promising results.

Hyperspectral face recognition with spatiospectral information fusion and PLS regression.

Hyperspectral imaging offers new opportunities for face recognition via improved discrimination along the spectral dimension. However, it poses new challenges, including low signal-to-noise ratio, interband misalignment, and high data dimensionality. Due to these challenges, the literature on hyperspectral face recognition is not only sparse but is limited to ad hoc dimensionality reduction techniques and lacks comprehensive evaluation. We propose a hyperspectral face recognition algorithm using a spatiospectral covariance for band fusion and partial least square regression for classification. Moreover, we extend 13 existing face recognition techniques, for the first time, to perform hyperspectral face recognition.We formulate hyperspectral face recognition as an image-set classification problem and evaluate the performance of seven state-of-the-art image-set classification techniques. We also test six state-of-the-art grayscale and RGB (color) face recognition algorithms after applying fusion techniques on hyperspectral images. Comparison with the 13 extended and five existing hyperspectral face recognition techniques on three standard data sets show that the proposed algorithm outperforms all by a significant margin. Finally, we perform band selection experiments to find the most discriminative bands in the visible and near infrared response spectrum.

Studies on Hyperspectral Face Recognition in Visible Spectrum With Feature Band Selection

This correspondence paper studies face recognition by using hyperspectral imagery in the visible light bands. The spectral measurements over the visible spectrum have different discriminatory information for the task of face identification, and it is found that the absorption bands related to hemoglobin are more discriminative than the other bands. Therefore, feature band selection based on the physical absorption characteristics of face skin is performed, and two feature band subsets are selected. Then, three methods are proposed for hyperspectral face recognition, including whole band (2D)2PCA, single band (2D)2PCA with decision level fusion, and band subset fusion-based (2D)2PCA. A simple yet efficient decision level fusion strategy is also proposed for the latter two methods. To testify the proposed techniques, a hyperspectral face database was established which contains 25 subjects and has 33 bands over the visible light spectrum (0.4-0.72 μm). The experimental results demonstrated that hyperspectral face recognition with the selected feature bands outperforms that by using a single band, using the whole bands, or, interestingly, using the conventional RGB color bands.