Classical Dimensionality Reduction Methods Research Articles

Comparison and Evaluation of Classical Dimensionality Reduction Methods

As one of the tasks of unsupervised learning, data dimensionality reduction is faced with the problem of a lack of evaluation methods. Based on this, three classical dimensionality reduction methods such as PCA, t-SNE and UMAP were selected as the research object in this paper. This article selected 5 three-classification datasets and used the three methods mentioned above to perform dimensionality reduction. This paper plotted 3D scatter graphs after dimensionality reduction to analyze the differentiation effect of the data on different categories of the target variable. Then the data after dimensionality reduction was classified using random forest model and the classification accuracy was obtained. According to the 3D scatter plots and the accuracy of random forest, it is found that PCA has a good dimensionality reduction effect on most of the selected datasets, and t-SNE has a relatively stable dimensionality reduction effect. In contrast, UMAP has good dimensionality reduction performance in some individual datasets but lacks stability. Overall, this paper proposes a dimensionality reduction evaluation method that combines scatter-plot visualization results and classification models, which can effectively predict the performances of the dimensionality reduction methods for a variety of datasets, thereby promoting the comparison and selection of dimensionality reduction methods in the field of unsupervised learning.

Unsupervised dimensionality reduction based on fusing multiple clustering results

The majority of the classical dimensionality reduction methods can be unified into a graph-embedding-based framework. A fixed graph constructed in a high-dimensional space has been extensively employed in the graph-embedding-based dimensionality reduction methods. However, a fixed graph often cannot characterize the structure of high-dimensional data owing to the curse of dimensionality. To solve this problem, we combine graph construction and dimensionality reduction into a coherent framework. Thus, the constructed graph can be updated dynamically in dimensionality reduction. In the existing methods based on the coherent framework, graphs are usually constructed by a type of neighborhood relationship and single clustering result. This study proposes an unsupervised dimensionality reduction method guided by fusing multiple clustering results. In the proposed method, multiple clustering results are first obtained by the k-means algorithm, and then a graph is constructed using a weighted co-association matrix of fusing the clustering results to capture data distribution information. Based on the graph, we present an objective function of combining graph construction and dimensionality reduction to implement mutual guidance between them. Numerical experiments on real data sets illustrate that the proposed method achieves significant improvement over some representative and state-of-the-art unsupervised dimensionality reduction methods.

Semi-supervised double sparse graphs based discriminant analysis for dimensionality reduction

Discriminant analysis (DA) is a well-known dimensionality reduction tool in pattern classification. With enough efficient labeled samples, the optimal projections could be found by maximizing the between-class scatter variance meanwhile minimizing the within-class scatter variance. However, the acquisition of label information is difficult in practice. So, semi-supervised discriminant analysis has attracted much attention in recent years, where both few labeled samples and many unlabeled samples are utilized during learning process. Sparse graph learned by sparse representation contains local structure information about data and is widely employed in dimensionality reduction. In this paper, semi-supervised double sparse graphs (sDSG) based dimensionality reduction is proposed, which considers both the positive and negative structure relationship of data points by using double sparse graphs. Aiming to explore the discriminant information among unlabeled samples, joint k nearest neighbor selection strategy is proposed to select pseudo-labeled samples which contain some precise discriminant information. In the following procedures, the data subset consisting of labeled samples and pseudo-labeled samples are used instead of the original data. Based on two different criterions, two sDSG based discriminant analysis methods are designed and denoted by sDSG-dDA (distance-based DA) and sDSG-rDA (reconstruction-based DA), which also use different strategies to reduce the effect of pseudo-labels’ inaccuracy. Finally, the experimental results both on UCI datasets and hyperspectral images validate the effectiveness and advantage of the proposed methods compared with some classical dimensionality reduction methods.

MiLDA: A graph embedding approach to multi-view face recognition

In a vast number of real-world face recognition applications, gallery and probe image sets are captured from different scenarios. For such multi-view data, face recognition systems often perform poorly. To tackle this problem, in this paper we propose a graph embedding framework, which can project the multi-view data into a common subspace of higher discriminability between classes. This framework can be readily utilized to extend classical dimensionality reduction methods to multi-view scenarios. Hence, by utilizing the framework for multi-view face recognition, we propose multi-view linear discriminant analysis (MiLDA). We also empirically demonstrate that, for several distinct multi-view face recognition scenarios, MiLDA has an excellent performance and outperforms many popular approaches.

Analysis of Low-Dimensional Radio-Frequency Impedance-Based Cardio-Synchronous Waveforms for Biometric Authentication

Over the past two decades, there have been a lot of advances in the field of pattern analyses for biomedical signals, which have helped in both medical diagnoses and in furthering our understanding of the human body. A relatively recent area of interest is the utility of biomedical signals in the field of biometrics, i.e., for user identification. Seminal work in this domain has already been done using electrocardiograph (ECG) signals. In this paper, we discuss our ongoing work in using a relatively recent modality of biomedical signals-a cardio-synchronous waveform measured using a Radio-Frequency Impedance-Interrogation (RFII) device for the purpose of user identification. Compared to an ECG setup, this device is noninvasive and measurements can be obtained easily and quickly. Here, we discuss the feasibility of reducing the dimensions of these signals by projecting onto various subspaces while still preserving interuser discriminating information. We compare the classification performance using classical dimensionality reduction methods such as principal component analysis (PCA), independent component analysis (ICA), random projections, with more recent techniques such as K-SVD-based dictionary learning. We also report the reconstruction accuracies in these subspaces. Our results show that the dimensionality of the measured signals can be reduced by 60 fold while maintaining high user identification rates.

Sparsity preserving discriminant analysis for single training image face recognition

Single training image face recognition is one of the main challenges to appearance-based pattern recognition techniques. Many classical dimensionality reduction methods such as LDA have achieved success in face recognition field, but cannot be directly used to the single training image scenario. Recent graph-based semi-supervised dimensionality reduction (SSDR) provides a feasible strategy to deal with such problem. However, most of the existing SSDR algorithms such as semi-supervised discriminant analysis (SDA) are locality-oriented and generally suffer from the following issues: (1) they need a large number of unlabeled training samples to estimate the manifold structure in data, but such extra samples may not be easily obtained in a given face recognition task; (2) they model the local geometry of data by the nearest neighbor criterion which generally fails to obtain sufficient discriminative information due to the high-dimensionality of face image space; (3) they construct the underlying adjacency graph (or data-dependent regularizer) using a fixed neighborhood size for all the sample points without considering the actual data distribution. In this paper, we develop a new graph-based SSDR algorithm called sparsity preserving discriminant analysis (SPDA) to address these problems. More specifically, (1) the graph in SPDA is constructed by sparse representation, and thus the local structure in data is automatically modeled instead of being manually predefined. (2) With the natural discriminative power of sparse representation, SPDA can remarkably improve recognition performance only resorting to very few extra unlabeled samples. (3) A simple ensemble strategy is developed to accelerate graph construction, which results in an efficient ensemble SPDA algorithm. Extensive experiments on both toy and real face data sets are provided to validate the feasibility and effectiveness of the proposed algorithm.