L1-norm-based kernel entropy components

Abstract

Kernel entropy component analysis (KECA) is a recently proposed dimensionality reduction approach, which has showed superiority in many pattern analysis algorithms previously based on principal component analysis (PCA). The optimized KECA (OKECA) is a state-of-the-art extension of KECA and can return projections retaining more expressive power than KECA. However, OKECA is not robust to outliers and has high computational complexities attributed to its inherent properties of L2-norm. To tackle these two problems, we propose a new variant of KECA, namely L1-norm-based KECA (L1-KECA) for data transformation and feature extraction. L1-KECA attempts to find a new kernel decomposition matrix such that the extracted features store the maximum information potential, which is measured by L1-norm. Accordingly, we present a greedy iterative algorithm which has much faster convergence than OKECA's. Additionally, L1-KECA retains OKECA's capability to obtain accurate density estimation with very few features (just one or two). Moreover, a new semi-supervised L1-KECA classifier is developed and employed into the data classification. Extensive experiments on different real-world datasets validate that our model is superior to most existing KECA-based and PCA-based approaches. Code has been also made publicly available.