Enhanced feature fusion through irrelevant redundancy elimination in intra-class and extra-class discriminative correlation analysis

Abstract

Feature fusion aims to provide enhancements of data authenticity in both traditional and deep learning pattern analysis. Canonical correlation analysis (CCA) based feature fusion is a main technique for exploring the mutual relationships of multiple feature sets. In traditional CCA-based feature fusion, the dimensionality of each feature set is usually first reduced using principal component analysis (PCA), linear discriminant analysis (LDA) etc. to ensure non-singularity and invertibility of covariance matrices. One issue with the above standard CCA-based feature fusion is that the reduced feature sets generated by PCA or LDA may neglect certain correlation information among different feature sets which is useful for CCA, and this in turn may degrade the following classification performance. Another issue is that most CCA fused features may still contain redundancies due to the correlation criterion. These redundancies may be relevant or irrelevant to class labels. The irrelevant redundancies may degrade the pattern recognition performance, while the relevant redundancies can make the pattern recognition system more robust. In this paper, we propose an enhanced feature fusion scheme through irrelevant redundancy elimination in intra-class and extra-class discriminative correlation analysis (IEDCA-IRE) addressing the above two issues. IEDCA-IRE explores the intra-class correlation including both the pairs-wise correlation like CCA-based feature fusion approaches and the correlation across different features within the same class. By incorporating kernelized IEDCA into minimum redundancy maximum relevance (mRMR) criterion, only the relevant redundancy is retained in the fused feature. Our proposed IEDCA-IRE can be used in unimodal feature fusion, multimodal feature fusion, fusion of deep features extracted from different deep neural network models as well as fusion of deep features and handcrafted features. Extensive experiments have proved its effectiveness.