Unsupervised Dimensionality Reduction Algorithm Research Articles

Robust Principal Component Analysis via Joint Reconstruction and Projection.

Principal component analysis (PCA) is one of the most widely used unsupervised dimensionality reduction algorithms, but it is very sensitive to outliers because the squared l2 -norm is used as distance metric. Recently, many scholars have devoted themselves to solving this difficulty. They learn the projection matrix from minimum reconstruction error or maximum projection variance as the starting point, which leads them to ignore a serious problem, that is, the original PCA learns the projection matrix by minimizing the reconstruction error and maximizing the projection variance simultaneously, but they only consider one of them, which imposes various limitations on the performance of model. To solve this problem, we propose a novel robust principal component analysis via joint reconstruction and projection, namely, RPCA-RP, which combines reconstruction error and projection variance to fully mine the potential information of data. Furthermore, we carefully design a discrete weight for model to implicitly distinguish between normal data and outliers, so as to easily remove outliers and improve the robustness of method. In addition, we also unexpectedly discovered that our method has anomaly detection capabilities. Subsequently, an effective iterative algorithm is explored to solve this problem and perform related theoretical analysis. Extensive experimental results on several real-world datasets and RGB large-scale dataset demonstrate the superiority of our method.

Structure-aware preserving projections with applications to medical image clustering

The application of dimensionality reduction (DR) in effectively handling high-dimensional data is becoming increasingly prominent. However, mainstream methods in projection learning exhibit a reliance on either global or local structures, resulting in the inability to effectively learn the discriminative projection. To address this issue, we propose a novel unsupervised DR algorithm, called Structure-Aware Preserving Projections (SAPP). Specifically, SAPP exploits a powerful combination of subspace clustering and graph construction to effectively capture the intrinsic structure of the data. It is able to find discriminative projections, which is mainly attributed to two key factors: (1) can capture the local structure of the data by constructing the nearest neighbor graph and introducing the grouping effect of the representation into projection learning; and (2) preserves the global structure and respects the complex relationship between data points by integrating least squares regression into the dimension reduction process. Moreover, SAPP proves to be a versatile framework that is easily extended to semi-supervised scenarios. Extensive experiments on medical image datasets confirm the effectiveness of our proposed method, showcasing superior clustering accuracy compared to state-of-the-art approaches.

Discrete Robust Principal Component Analysis via Binary Weights Self-Learning.

Principal component analysis (PCA) is a typical unsupervised dimensionality reduction algorithm, and one of its important weaknesses is that the squared l2 -norm cannot overcome the influence of outliers. Existing robust PCA methods based on paradigm have the following two drawbacks. First, the objective function of PCA based on the l1 -norm has no rotational invariance and limited robustness to outliers, and its solution mostly uses a greedy search strategy, which is expensive. Second, the robust PCA based on the l2,1 -norm and the l2,p -norm is essential to learn probability weights for data, which only weakens the influence of outliers on the learning projection matrix and cannot be completely eliminated. Moreover, the ability to detect anomalies is also very poor. To solve these problems, we propose a novel discrete robust principal component analysis (DRPCA). Through self-learning binary weights, the influence of outliers on the projection matrix and data center estimation can be completely eliminated, and anomaly detection can be directly performed. In addition, an alternating iterative optimization algorithm is designed to solve the proposed problem and realize the automatic update of binary weights. Finally, our proposed model is successfully applied to anomaly detection applications, and experimental results demonstrate that the superiority of our proposed method compared with the state-of-the-art methods.

Max–Min Robust Principal Component Analysis

Principal Component Analysis (PCA) is a powerful unsupervised dimensionality reduction algorithm, which uses squared ℓ2-norm to cleverly connect reconstruction error and projection variance, and those improved PCA methods only consider one of them, which limits their performance. To alleviate this problem, we propose a novel Max–Min Robust Principal Component Analysis via binary weight, which ingeniously combines reconstruction error and projection variance to learn projection matrix more accurately, and uses ℓ2-norm as evaluation criterion to make the model rotation invariant. In addition, we design binary weight to remove outliers to improve the robustness of model and obtain the ability of anomaly detection. Subsequently, we exploit an efficient iterative optimization algorithm to solve this problem. Extensive experimental results show that our model outperforms related state-of-the-art PCA methods.

The DataScope: A Mixed-Initiative Architecture for Data Labeling

Machine learning promises many advantages, but achieving these promises requires methods that smooth the interaction between humans and machine learning. Machine learning systems require meticulous training on large, labeled datasets. Labeling data is a tedious expensive process, and many times requires complex human-judgment skills. Mixed-initiative designs divide the labor between the artificial agent and the human to make the task at hand more efficient and effective. This paper proposes a mixed-initiative method for efficient coding of video and image data in general and emotion data in particular. We integrate an unsupervised dimensionality reduction algorithm and the R Shiny platform to develop an interactive method that leverages human expertise to label the data more efficiently and effectively. The method, through the interactive web tool, allows the user to explore the data interactively, examine similarities and dissimilarities in the data, and label clusters of many images and video frames at once. The combination of the unsupervised learning algorithm and the R Shiny platform enables interactive exploration and annotation of high-dimensional, complicated data. This method can be used to annotate large data sets faster and can advance research in machine vision.

MO004: Hidden Patterns of T Cell and B Cell Phenotypes seen Through Machine Learning Algorithms in End-Stage Renal Disease

Abstract BACKGROUND AND AIMS End-stage renal disease (ESRD) is essentially a chronic inflammatory state and, consequently, it engenders detrimental effects on the immune system. Due to the specific uremic environment, multiple phenotypic lymphocyte alterations are described, similar but not identical to ageing. We aimed to evaluate these changes through simple unsupervised dimensionality reduction algorithms in order to reveal unique phenotypical lymphocyte patterns in ESRD patients. METHOD A wide panel of senescent and exhaustion-related lymphocyte markers, including CD45RA, CCR7, CD31, CD28, CD57, and PD1 on T cells and CD27 and IgD on B cells, was assessed by flow cytometry in 30 ESRD patients and 20 healthy controls of similar age, sex and ethnicity. The resulting immunological multidimensional phenotype was projected at lower dimensions using two algorithms: principal component analysis (PCA) and Uniform Manifold Approximation and Projection (UMAP). RESULTS The plane defined by the two first eigenvectors of PCA showed two clusters, patients and controls, with PCA variable loadings of non-senescent markers pointing towards the controls' centroid. Indeed, ‘naïve’ lymphocytes were reduced in ESRD patients compared with controls [CD4+CD45RA+CCR7+200(150–328) versus 426(260–585) cells/μL, respectively; P = .001] and [CD19+IgD+CD27–54(26–85) versus 130(83–262) cells/μL, respectively; P < .0001]. Also, PCA projections of the multidimensional ESRD immune phenotype suggested a more senescent phenotype in haemodialysis compared with haemodiafiltration treated patients. Finally, clustering based on UMAP projections revealed three distinct patients groups (UMAP 1–3), exhibiting gradual changes for naive, senescent and exhausted lymphocyte markers. Out of these, the UMAP 1 group was characterized by a predominance of CD8 senescent T cell subsets, in contrast, to groups UMAP 2 and 3, which showed a gradual decrease of all T cell subsets, in comparison to healthy controls. These groups were found to differ in the type of dialyzer used, with polysulfone or derivatives alone more often prescribed in UMAP 2–3 patients (13 out of 17) compared with 4 out of 13 in UMAP 1, P = .012. CONCLUSION Simple machine learning algorithms may help to unravel hidden lymphocyte markers and define patterns characteristic of ESRD. Moreover, significant connections of immune changes with dialysis methods and dialyzers were revealed and described.

Learning new physical descriptors from reduced-order analysis of bubble dynamics in boiling heat transfer

Understanding bubble dynamics during boiling is challenging due to the drastic changes in system parameters, such as nucleation, bubble morphology, temperature, and pressure. In this study, principal component analysis (PCA), an unsupervised dimensionality reduction algorithm, is used to extract new physical descriptors of boiling heat transfer from pool boiling experimental images without labeling and training. The dominant frequency and amplitude of the time-series principal components (PCs) are analyzed, where the first few dominant PCs are used to approximate the instantaneous bubble morphologies, drastically reducing the data dimensions. The results show that the dominant frequency and amplitude can be used as new physical descriptors to distinguish different boiling regimes. The dominant frequency of the first PC is found to increase with heat flux in the discrete bubble regime until it reaches a peak and then decreases with heat flux in the bubble interference and coalescence regime, where the former is believed to be associated with the increase in bubble nucleation sites and the latter is associated with the bubble coalescence during pool boiling. The dominant frequency and amplitude extracted from the present unsupervised learning are qualitatively compared to the bubble count and size extracted from a supervised deep-learning algorithm, and the approach appears highly robust over multiple datasets and heater surfaces. To predict future boiling states for mitigating boiling crises, bidirectional long short-term memory (BiLSTM) neural network is used to estimate the future variations of PCs and hence the bubble dynamics, from time-series PCs. The PCA-BiLSTM models predict reduced-order bubble images well and show significantly higher prediction accuracy compared to the Convolutional-LSTM.

Kernel alignment unsupervised discriminative dimensionality reduction

The performance of most existing adaptive graph learning methods, which adjust data similarity matrix according to the data representation, depends on the hypotheses that the data representation is a good indicator of the underlying data structure. However, this hypothesis is not always applicable when dealing with high dimensional data. In this paper, we propose a novel kernel alignment unsupervised discriminative dimensionality reduction (KaUDDR) algorithm. By integrating adaptive graph learning and feature learning into a joint learning framework, graph construction and dimensionality reduction are conducted simultaneously to guarantee the optimality of graph for feature learning in the proposed algorithm. Data kernel and similarity indicator kernel are defined by learned graph and the projected data in a low-dimensional subspace, a compact and discriminative data representation in the projected subspace is obtained by means of kernel alignment to explore the consistency between the projected data kernel and similarity indicator kernel. Experimental results on dimensionality reduction as well as clustering show that our method consistently outperforms the related unsupervised dimensionality reduction algorithm.

A Clustering Analysis of Renaissance Polyphony Using State-Space Models

In this paper, we propose a novel clustering method to represent and compare works of symbolic music, in particular the music in the database of the Josquin Research Project. We also study the results of our methods on the database as a whole, as well as the results for some specific cases. The methods that we propose revolve around modelling the activity of the twelve pitch classes (i.e., chromae) over time. We suggest that by modelling this activity, we can capture the harmonic progressions in the music. By converting the models to a vector representation, we can analyse the works in the dataset using clustering techniques from machine learning. These techniques can be used to group, visualize, and classify works in the corpus. This way, we suggest attributions for insecurely attributed pieces in the dataset, based solely on the harmonic progressions contained in the songs. We also apply our final classification models to the unica of the Leuven Chansonnier, to give the first suggestions for the attribution of these works from the world of computational musicology. We then compare our methods to some existing methods for analyzing symbolic music, and find that our method is slightly worse at identifying composers of Renaissance works. This result was not completely unexpected, as the features we extract are based only the musical progressions present in the music. Finally, we use the best vector representation we devised together with unsupervised dimensionality reduction algorithms to create a scatterplot visualization of the entire Josquin Research Project dataset.

A novel unsupervised Globality-Locality Preserving Projections in transfer learning

In this paper, a novel unsupervised dimensionality reduction algorithm, unsupervised Globality-Locality Preserving Projections in Transfer Learning (UGLPTL) is proposed, based on the conventional Globality-Locality Preserving dimensionality reduction algorithm (GLPP) that does not work well in real-world Transfer Learning (TL) applications. In TL applications, one application (source domain) contains sufficient labeled data, but the related application contains only unlabeled data (target domain). Compared to the existing TL methods, our proposed method incorporates all the objectives, such as minimizing the marginal and conditional distributions between both the domains, maximizing the variance of the target domain, and performing Geometrical Diffusion on Manifolds, all of which are essential for transfer learning applications. UGLPTL seeks a projection vector that projects the source and the target domains data into a common subspace where both the labeled source data and the unlabeled target data can be utilized to perform dimensionality reduction. Comprehensive experiments have verified that the proposed method outperforms many state-of-the-art non-transfer learning and transfer learning methods on two popular real-world cross-domain visual transfer learning data sets. Our proposed UGLPTL approach achieved 82.18% and 87.14% mean accuracies over all the tasks of PIE Face and Office-Caltech data sets, respectively.

Dimensionality Reduction of Hyperspectral Imagery Based on Spatial-Spectral Manifold Learning.

The graph embedding (GE) methods have been widely applied for dimensionality reduction of hyperspectral imagery (HSI). However, a major challenge of GE is how to choose the proper neighbors for graph construction and explore the spatial information of HSI data. In this paper, we proposed an unsupervised dimensionality reduction algorithm called spatial-spectral manifold reconstruction preserving embedding (SSMRPE) for HSI classification. At first, a weighted mean filter (WMF) is employed to preprocess the image, which aims to reduce the influence of background noise. According to the spatial consistency property of HSI, SSMRPE utilizes a new spatial-spectral combined distance (SSCD) to fuse the spatial structure and spectral information for selecting effective spatial-spectral neighbors of HSI pixels. Then, it explores the spatial relationship between each point and its neighbors to adjust the reconstruction weights to improve the efficiency of manifold reconstruction. As a result, the proposed method can extract the discriminant features and subsequently improve the classification performance of HSI. The experimental results on the PaviaU and Salinas hyperspectral data sets indicate that SSMRPE can achieve better classification results in comparison with some state-of-the-art methods.

Supervised dynamic mode decomposition via multitask learning

Understanding dynamical systems by extracting spatiotemporal patterns from data is fundamental in a variety of fields of engineering and science. Dynamic mode decomposition (DMD) has recently attracted attention in these fields as a way of obtaining a global modal description of a nonlinear dynamical system from data, without requiring explicit prior knowledge. However, DMD is in principle an unsupervised dimensionality reduction algorithm; it is not endowed with the mechanism to utilize label information even if a set of data with different labels is given. In this paper, we propose the algorithm that incorporates label information into DMD via multitask learning by solving sparse-group Lasso. To this end, we estimate sparse weights over dynamic modes in a label-wise manner by regarding data with different labels as different tasks. Modal descriptions estimated by this approach share a part of the global modes, resulting in the extraction of label-specific and common (or mixed) dynamical structures, which could be useful in understanding mechanisms in the spatiotemporal behavior behind data. We investigate the empirical performance using synthetic and real-world datasets, and validate that our algorithm can extract and visualize common and label-specific spatiotemporal structures.

Singular Value Decomposition Projection for solving the small sample size problem in face recognition

Numerous dimensionality reduction methods have achieved impressive performance in face recognition field due to their potential to exploit the intrinsic structure of images and to enhance the computational efficiency. However, the FR methods based on the existing dimensionality reduction often suffer from small sample size (SSS) problems, where the sample dimensionality is larger than the number of training samples per subject. In recent years, Sparse Representation based Classification (SRC) has been demonstrated to be a powerful framework for robust FR. In this paper, a novel unsupervised dimensionality reduction algorithm, called Singular Value Decomposition Projection (SVDP), is proposed to better fit SRC for handling the SSS problems in FR. In SVDP, a weighted linear transformation matrix is derived from the original data matrix via Singular Value Decomposition. The projection obtained in this way is row-orthonormal and it has some good properties. It makes the solution be robust to small perturbations contained in the data and has better ability to represent various signals. Thus, SVDP could better preserve the discriminant information of the data. Based on SVDP, a novel face recognition method SVDP-SRC is designed to enable SRC to achieve better performance via low-dimensional representation of faces. The experiments carried out with some simulated data show that SVDP achieves higher recovery accuracy than several other dimensionality reduction methods. Moreover, the results obtained on three standard face databases demonstrate that SVDP-SRC is quite effective to handle the SSS problems in terms of recognition accuracy.

An unsupervised data projection that preserves the cluster structure

In this paper we propose a new unsupervised dimensionality reduction algorithm that looks for a projection that optimally preserves the clustering data structure of the original space. Formally we attempt to find a projection that maximizes the mutual information between data points and clusters in the projected space. In order to compute the mutual information, we neither assume the data are given in terms of distributions nor impose any parametric model on the within-cluster distribution. Instead, we utilize a non-parametric estimation of the average cluster entropies and search for a linear projection and a clustering that maximizes the estimated mutual information between the projected data points and the clusters. The improved performance is demonstrated on both synthetic and real world examples.