A Pairwise Covariance-Preserving Projection Method for Dimension Reduction

Abstract

Dimension reduction is critical in many areas of pattern classification and machine learning and many discriminant analysis algorithms have been proposed. In this paper, a Pairwise Covariance-preserving Projection Method (PCPM) is proposed for dimension reduction. PCPM maximizes the class discrimination and also preserves approximately the pairwise class covariances. The optimization involved in PCPM can be solved directly by eigenvalues decomposition. Our theoretical and empirical analysis reveals the relationship between PCPM and Linear Discriminant Analysis (LDA), Sliced Average Variance Estimator (SAVE), Heteroscedastic Discriminant Analysis (HDA) and Covariance preserving Projection Method (CPM). PCPM can utilize class mean and class covariance information at the same time. Furthermore, pairwise weight scheme can be incorporated naturally with the pairwise summarization form. The proposed methods are evaluated by both synthetic and real-world datasets.