Image set classification based on synthetic examples and reverse training

Abstract

Classification based on image sets has recently attracted increasing interests in computer vision and pattern recognition community. It finds numerous applications in real-life scenarios, such as classification from surveillance videos, multi-view camera networks, and personal albums. Image set based face classification highly depends on the consistency and coverage of the poses and view point variations of a subject in gallery and probe sets. This paper explores a synthetic method to create the unseen face features in the database, thus achieving better performance of image set based face recognition. By considering the high symmetry of human faces, multiple synthetic instances are virtually generated to make up the missing parts, so as to enrich the variety of the database. With respect to the classification framework, we resort to reverse training due to its high efficiency and accuracy. The performance of the proposed approach, Synthetic Examples based Reverse Training (SERT), has been fully evaluated on Honda/UCSD, CMU Mobo and YouTube Celebrities, three benchmark datasets comprising facial image sequences. Extensive comparisons with the other state-of-the-art methods have corroborated the superiority of our approach.