Supervised Dimensionality Reduction Approach Research Articles

A Boundary Aware WiFi Localization Scheme Based on UMAP and KNN

In this letter, a novel localization scheme is reported to improve WiFi fingerprint-based localization via dimensionality reduction and an adaptive fingerprint matching algorithm. Specifically, an experimental analysis is firstly carried out to investigate the quality of radio maps. Then, the advanced uniform manifold approximation and projection (UMAP) algorithm is applied to refine the mapping relations from a fingerprint space to a location space in a supervised dimensionality reduction approach, which significantly improves the quality of radio maps. In addition, to further enhance the localization accuracy, boundary aware KNN (B-KNN) is proposed to adaptively deal with fingerprint matching according to inner area or boundary area. Extensive experiments are conducted in a real scenario with the area of nearly <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$1000~m^{2}$ </tex-math></inline-formula> , and a thorough comparison with several existing popular methods illustrates that the proposed scheme can significantly improve the localization accuracy on average by 33.3%.

A supervised take on dimensionality reduction via hybrid subset selection

Amouzgar et al. present HSS-LDA, a supervised dimensionality reduction approach for single-cell data that outperforms existing unsupervised techniques. They couple hybrid subset selection to linear discriminant analysis and identify interpretable linear combinations of predictors that best separate predefined biological groups.

Adaptive Local Linear Discriminant Analysis

Dimensionality reduction plays a significant role in high-dimensional data processing, and Linear Discriminant Analysis (LDA) is a widely used supervised dimensionality reduction approach. However, a major drawback of LDA is that it is incapable of extracting the local structure information, which is crucial for handling multimodal data. In this article, we propose a novel supervised dimensionality reduction method named Adaptive Local Linear Discriminant Analysis (ALLDA), which adaptively learns a k -nearest neighbors graph from data themselves to extract the local connectivity of data. Furthermore, the original high-dimensional data usually contains noisy and redundant features, which has a negative impact on the evaluation of neighborships and degrades the subsequent classification performance. To address this issue, our method learns the similarity matrix and updates the subspace simultaneously so that the neighborships can be evaluated in the optimal subspaces where the noises have been removed. Through the optimal graph embedding, the underlying sub-manifolds of data in intra-class can be extracted precisely. Meanwhile, an efficient iterative optimization algorithm is proposed to solve the minimization problem. Promising experimental results on synthetic and real-world datasets are provided to evaluate the effectiveness of proposed method.

Joint Principal Component and Discriminant Analysis for Dimensionality Reduction.

Linear discriminant analysis (LDA) is the most widely used supervised dimensionality reduction approach. After removing the null space of the total scatter matrix St via principal component analysis (PCA), the LDA algorithm can avoid the small sample size problem. Most existing supervised dimensionality reduction methods extract the principal component of data first, and then conduct LDA on it. However, "most variance" is very often the most important, but not always in PCA. Thus, this two-step strategy may not be able to obtain the most discriminant information for classification tasks. Different from traditional approaches which conduct PCA and LDA in sequence, we propose a novel method referred to as joint principal component and discriminant analysis (JPCDA) for dimensionality reduction. Using this method, we are able to not only avoid the small sample size problem but also extract discriminant information for classification tasks. An iterative optimization algorithm is proposed to solve the method. To validate the efficacy of the proposed method, we perform extensive experiments on several benchmark data sets in comparison with some state-of-the-art dimensionality reduction methods. A large number of experimental results illustrate that the proposed method has quite promising classification performance.

Discriminative Dimensionality Reduction for Multi-Dimensional Sequences.

Since the observables at particular time instants in a temporal sequence exhibit dependencies, they are not independent samples. Thus, it is not plausible to apply i.i.d. assumption-based dimensionality reduction methods to sequence data. This paper presents a novel supervised dimensionality reduction approach for sequence data, called Linear Sequence Discriminant Analysis (LSDA). It learns a linear discriminative projection of the feature vectors in sequences to a lower-dimensional subspace by maximizing the separability of the sequence classes such that the entire sequences are holistically discriminated. The sequence class separability is constructed based on the sequence statistics, and the use of different statistics produces different LSDA methods. This paper presents and compares two novel LSDA methods, namely M-LSDA and D-LSDA. M-LSDA extracts model-based statistics by exploiting the dynamical structure of the sequence classes, and D-LSDA extracts the distance-based statistics by computing the pairwise similarity of samples from the same sequence class. Extensive experiments on several different tasks have demonstrated the effectiveness and the general applicability of the proposed methods.

Moments discriminant analysis for supervised dimensionality reduction

Most of the well-known supervised dimensionality reduction methods assume unimodal or Gaussian likelihoods, which may not be appropriate in the real life applications. In this manuscript, we introduce a novel supervised dimensionality reduction approach, moments discriminant analysis, which models linear relationships between the high-dimensional input space and a low-dimensional space by maximizing the discrimination between second order raw moments of different classes to improve the generalization capability of a classifier. Unlike the state-of-the-art methods, moments discriminant analysis is intended to accommodate data distributions that may be multimodal and non-Gaussian. Initially, experiments using synthetic random data (generated from different probability distributions) are performed to prove the efficiency of the proposed method for multimodal and non-Gaussian data with the help of five separability measures. Also, extensive experimental results on UCI machine learning repository and image retrieval on WANG and MIT (Oliva and Torralba) databases are carried out in order to exhibit the effectiveness of moments discriminant analysis over the state-of-the-art methods.

Dimensionality reduction by supervised neighbor embedding using laplacian search.

Dimensionality reduction is an important issue for numerous applications including biomedical images analysis and living system analysis. Neighbor embedding, those representing the global and local structure as well as dealing with multiple manifolds, such as the elastic embedding techniques, can go beyond traditional dimensionality reduction methods and find better optima. Nevertheless, existing neighbor embedding algorithms can not be directly applied in classification as suffering from several problems: (1) high computational complexity, (2) nonparametric mappings, and (3) lack of class labels information. We propose a supervised neighbor embedding called discriminative elastic embedding (DEE) which integrates linear projection matrix and class labels into the final objective function. In addition, we present the Laplacian search direction for fast convergence. DEE is evaluated in three aspects: embedding visualization, training efficiency, and classification performance. Experimental results on several benchmark databases present that the proposed DEE exhibits a supervised dimensionality reduction approach which not only has strong pattern revealing capability, but also brings computational advantages over standard gradient based methods.

(2D)2LDALPP: A Novel Approach to Face Recognition

This paper presents a novel supervised dimensionality reduction approach for facial feature extraction called (2D)2LDALPP. The proposed (2D)2LDALPP method effectively combines alternative 2DLDA with alternative 2DLPP. The feature extraction is split into two steps: firstly, the column directional information is extracted by applying alternative 2DLDA; secondly, the feature matrix is inversed and alternative 2DLPP is used to extract the row directional information. The advantage of the method lies in the compression of the facial image in two different directions and the fact that the dimension of the feature matrix is low. At the same time, because 2DLDA is a supervised learning method, the proposed method not only preserves the manifold structure of the samples but also contains the label information of the classes. Experimental results on the Feret, ORL, and Yale databases show that the proposed method is effective.