Multisensor fusion of visual and thermal images for human face identification using different SVM kernels

Abstract

In this paper we present a novel method of face identification using different levels of pixel fusion (e.g. ratios for pixel information taken from the visual and thermal images are, 2:3, 1:1, 3:2 and 7:3) and classification of fused images using different kernels of Support Vector Machine (SVM). Visual imagery has been broadly used in face identification systems, but these are very sensitive to illumination changes. This limitation has been overcome by the Infrared (IR) spectrum that provides simpler and more robust solution to boost the identification performance in uncontrolled environments and deliberate attempts to obscure identity. But IR imagery is sensitive to temperature changes in the surrounding environment and variations in the heat patterns of the face and it is opaque to glass. All these facts degrade the face identification efficiency. This drove us to fuse information from both visual and thermal spectra, which have the potential to improve face identification performance as fusion of thermal and visual images provide improved images with more compact information. Once we get fused images those are reduced in dimension using Eigenvalue Decomposition based Candid Co-variance free Incremental Principal Component Analysis (EVD-CCIPCA) and these reduced fused images are classified using the three different kernels of SVM. The three kernels used here are: linear, polynomial and gaussian RBF. SVM is primarily a classifier method that performs classification tasks by constructing hyperplanes in a multidimensional space that separates cases of different class labels. In this paper, we have used multiclass SVM to carry out identification on face images and Quadratic Programming (QP) optimization method to train the SVM. For experiments, IRIS Thermal/Visual Face Database is used. Experimental results show that 97.28% is the highest average success rate achieved on the fused images of 70% visual and 30% thermal images using the linear kernel. However, the highest success rate of 100% is achieved for classes 4 and 10 in several cases.