Rank Pooling for Image set Based Face Recognition

Abstract

Image set based face recognition is a very relevant and interesting problem in computer vision. Set based image recognition in general and face recognition in particular is composed of two major components-a) Feature extraction and b) Feature aggregation. Image set based face recognition aims at capturing relevant information from all the images to infer the identity of a person thereby being robust to a variety of situations such as pose, illumination effects, etc. In this paper, we propose a novel approach to feature aggregation for face recognition using multiple images. We build on a robust feature embedding network which maximizes the clustering coefficient in the embedding space and use a novel rank pooling algorithm to infer the identity of a person from an image set. We compare the performance of the proposed algorithm with state-of-the-art techniques in the literature on YouTube Celebrities data set(YTC), YouTube Faces data set (YTF). We show that the proposed algorithm gives a significant improvement in recognition accuracy relative to the existing algorithms in face recognition using image sets.