A brief survey on multispectral face recognition and multimodal score fusion

Abstract

In this paper, we explore and compare four face recognition methods and their performance with multispectral face images, and further investigate the performance improvement using multimodal score fusion. The four face recognition methods include three classical methods, PCA, LDA and EBGM (elastic bunch graph matching), and one new method, FPB (face pattern byte). The FPB method actually extracts orientational facial features by Gabor wavelet transform and uses Hamming distance for face identification. When the multispectral images from the same subject are available, the identification accuracy and reliability can be significantly enhanced using score fusion. Four score fusion methods, mean fusion, LDA fusion, KNN (k-nearest neighbor) fusion, and HMM (hidden Markov model) fusion are implemented and compared. Our experiments are conducted with the ASUMS face database that currently consists of two-band images (visible and thermal) from 96 subjects. We compare the identification performance of applying the four recognition methods to the two-band face images, and compare the fusion performance of combing the multiple scores from different methods (matcher) and from different bands (modality) of face images. The experimental results show that the face identification rate can achieve 100% when fusing two FPB scores from two-band face images; overall, the FPB method performs the best; the score modality is a key factor in biometric score fusion; when the number of score modalities is fixed, the fusion method becomes next important factor to score fusion; and the HMM fusion is the most reliable score fusion method.