Learning Discriminative Canonical Correlations for Object Recognition with Image Sets

Abstract

We address the problem of comparing sets of images for object recognition, where the sets may represent arbitrary variations in an object’s appearance due to changing camera pose and lighting conditions. The concept of Canonical Correlations (also known as principal angles) can be viewed as the angles between two subspaces. As a way of comparing sets of vectors or images, canonical correlations offer many benefits in accuracy, efficiency, and robustness compared to the classical parametric distribution-based and non-parametric sample-based methods. Here, this is demonstrated experimentally for reasonably sized data sets using existing methods exploiting canonical correlations. Motivated by their proven effectiveness, a novel discriminative learning over sets is proposed for object recognition. Specifically, inspired by classical Linear Discriminant Analysis (LDA), we develop a linear discriminant function that maximizes the canonical correlations of within-class sets and minimizes the canonical correlations of between-class sets. The proposed method significantly outperforms the state-of-the-art methods on two different object recognition problems using face image sets with arbitrary motion captured under different illuminations and image sets of five hundred general object categories taken at different views.