Clustering High-dimensional Data Research Articles

Deep Multi-Constraint Soft Clustering Analysis for Single-Cell RNA-Seq Data via Zero-Inflated Autoencoder Embedding.

Clustering cells into subgroups plays a critical role in single cell-based analyses, which facilitates to reveal cell heterogeneity and diversity. Due to the ever-increasing scRNA-seq data and low RNA capture rate, it has become challenging to cluster high-dimensional and sparse scRNA-seq data. In this study, we propose a single-cell Multi-Constraint deep soft K-means Clustering(scMCKC) framework. Based on zero-inflated negative binomial (ZINB) model-based autoencoder, scMCKC constructs a novel cell-level compactness constraint by considering association between similar cell, to emphasize the compactness between clusters. Besides, scMCKC utilizes pairwise constraint encoded by prior information to guide clustering. Meanwhile, a weighted soft K-means algorithm is leveraged to determine the cell populations, which assigns the label based on affinity between data and clustering center. Experiments on eleven scRNA-seq datasets demonstrate that scMCKC is superior to the state-of-the-art methods and notably improves cluster performance. Moreover, we validate the robustness on human kidney dataset, which demonstrates that scMCKC exhibits comprehensively excellent performance on clustering analysis. The ablation study on eleven datasets proves that the novel cell-level compactness constraint is conductive to the clustering results.

HSCFC: High-dimensional streaming data clustering algorithm based on feedback control system

At present, few studies have considered the clustering analysis of high-dimensional data collected by many sensors in a real-time streaming environment. Existing clustering analysis algorithms for high-dimensional data are primarily based on batch processing models, and most of them cannot meet the requirements of incremental high-dimensional data streams, which are extremely common in practical applications. To address the aforementioned problems, this paper focuses on the study of high-dimensional data clustering based on stream processing, and proposes a high-dimensional data stream clustering algorithm based on a feedback control system, which comprises three stages: window principal component analysis, feedback stream clustering, and feedback controller. The classic exponentially weighted attenuation function is used in the window principal component analysis to avoid concept drift in the data stream, and incremental feature extraction executes through the sliding window to improve the iterative efficiency of the data in the window. To minimize the errors caused by variability in projection angles during dimensionality reduction, a feedback stream clustering stage is designed with alternating iterations of window clustering and cluster aggregation. Aiming at the problems caused by manually adjusting the hyperparameters used in high-dimensional data stream clustering, a feedback controller is developed to adjust the hyperparameters in the two other stages by analyzing the clustering results in real time and using a discriminant score to adopt corresponding feedback strategies. The experimental comparisons between the proposed and the traditional algorithms on multiple datasets demonstrate the effectiveness of the former.

Marigold: Efficientk-Means Clustering in High Dimensions

How can we efficiently and scalably cluster high-dimensional data? Thek-means algorithm clusters data by iteratively reducing intra-cluster Euclidean distances until convergence. While it finds applications from recommendation engines to image segmentation, its application to high-dimensional data is hindered by the need to repeatedly compute Euclidean distances among points and centroids. In this paper, we propose Marigold (k-means for high-dimensional data), a scalable algorithm fork-means clustering in high dimensions. Marigold prunes distance calculations by means of (i) a tight distance-bounding scheme; (ii) a stepwise calculation over a multiresolution transform; and (iii) exploiting the triangle inequality. To our knowledge, such an arsenal of pruning techniques has not been hitherto applied tok-means. Our work is motivated by time-critical Angle-Resolved Photoemission Spectroscopy (ARPES) experiments, where it is vital to detect clusters among high-dimensional spectra in real time. In a thorough experimental study with real-world data sets we demonstrate that Marigold efficiently clusters high-dimensional data, achieving approximately one order of magnitude improvement over prior art.

CoIn: Correlation Induced Clustering for Cognition of High Dimensional Bioinformatics Data.

Analysis of high dimensional biomedical data such as microarray gene expression data and mass spectrometry images, is crucial to provide better medical services including cancer subtyping, protein homology detection,etc. Clustering is a fundamental cognitive task which aims to group unlabeled data into multiple clusters based on their intrinsic similarities. The K-means algorithm is one of the most widely used clustering heuristics that aims at grouping the data objects into meaningful clusters such that the sum of squared Euclidean distances within each cluster is minimized. Its conceptual simplicity and computational efficiency make it easy to be used for wide applications of different data types. However, all features of data in K-means are considered equally in relevance, which distorts the performance when clustering high-dimensional data such as microarray gene expression data, mass spectrometry images, where there exist many redundant variables and correlated variables. In this paper, we propose a new correlation induced clustering, CoIn, to capture complex correlations among high dimensional data and guarantee the correlation consistency within each cluster. We evaluate the proposed method on a high dimensional mass spectrometry dataset of liver cancer tumor to explore the metabolic differences on tissues and discover the intra-tumor heterogeneity (ITH). By comparing the results of baselines and ours, it has been found that our method produces more explainable and understandable results for clinical analysis, which demonstrates the proposed clustering paradigm has the potential with application to knowledge discovery in high dimensional bioinformatics data.

Subspace Clustering in High-Dimensional Data Streams: A Systematic Literature Review

Clustering high dimensional data is challenging as data dimensionality increases the distance between data points, resulting in sparse regions that degrade clustering performance. Subspace clustering is a common approach for processing high-dimensional data by finding relevant features for each cluster in the data space. Subspace clustering methods extend traditional clustering to account for the constraints imposed by data streams. Data streams are not only high-dimensional, but also unbounded and evolving. This necessitates the development of subspace clustering algorithms that can handle high dimensionality and adapt to the unique characteristics of data streams. Although many articles have contributed to the literature review on data stream clustering, there is currently no specific review on subspace clustering algorithms in high-dimensional data streams. Therefore, this article aims to systematically review the existing literature on subspace clustering of data streams in high-dimensional streaming environments. The review follows a systematic methodological approach and includes 18 articles for the final analysis. The analysis focused on two research questions related to the general clustering process and dealing with the unbounded and evolving characteristics of data streams. The main findings relate to six elements: clustering process, cluster search, subspace search, synopsis structure, cluster maintenance, and evaluation measures. Most algorithms use a two-phase clustering approach consisting of an initialization stage, a refinement stage, a cluster maintenance stage, and a final clustering stage. The density-based top-down subspace clustering approach is more widely used than the others because it is able to distinguish true clusters and outliers using projected micro-clusters. Most algorithms implicitly adapt to the evolving nature of the data stream by using a time fading function that is sensitive to outliers. Future work can focus on the clustering framework, parameter optimization, subspace search techniques, memory-efficient synopsis structures, explicit cluster change detection, and intrinsic performance metrics. This article can serve as a guide for researchers interested in high-dimensional subspace clustering methods for data streams.

Parallel progressive-based inductive subspace and fuzzy-based firefly algorithm for high ensemble data clustering

Recently, the framework is proposed with techniques such as random subspace and constraint propagation for handling high dimensional data ensemble clustering. Huge dataset clustering is difficult for conventional sequential clustering methods since it requires higher computation time. Distributed parallel processing and methods are consequently useful towards attaining results and scalability constraints of clustering huge datasets. So, in this work, the parallel progressive-based inductive subspace ensemble clustering (PPISEC) algorithm is introduced with the concept of MapReduce (MR) to perform high dimensional data clustering. Depending on micro-clusters and correspondence relative, the clustering method is designed which is easily parallelised via MR and completed in moderately with a small number of MR rounds. However, in the PPISEC algorithm, the centroid values are selected with the help of an improved support vector machine (ISVM) classifier. Thus, the incremental ensemble member chosen (IEMC) progression is performed with fuzzy-based firefly algorithm (FFA), and the normalised cut algorithm is established to accomplish high dimensional data clustering. The outcome shows that, the proposed PPISEC framework, which performs well on three benchmark samples by utilising high dimensionality and enhanced the results than the conventional clustering ensemble approaches.

Multiview Spectral Clustering of High-Dimensional Observational Data

Multiview Spectral Clustering of High-Dimensional Observational Data

A Fast Hybrid Feature Selection Based on Correlation-Guided Clustering and Particle Swarm Optimization for High-Dimensional Data.

The "curse of dimensionality" and the high computational cost have still limited the application of the evolutionary algorithm in high-dimensional feature selection (FS) problems. This article proposes a new three-phase hybrid FS algorithm based on correlation-guided clustering and particle swarm optimization (PSO) (HFS-C-P) to tackle the above two problems at the same time. To this end, three kinds of FS methods are effectively integrated into the proposed algorithm based on their respective advantages. In the first and second phases, a filter FS method and a feature clustering-based method with low computational cost are designed to reduce the search space used by the third phase. After that, the third phase applies oneself to finding an optimal feature subset by using an evolutionary algorithm with the global searchability. Moreover, a symmetric uncertainty-based feature deletion method, a fast correlation-guided feature clustering strategy, and an improved integer PSO are developed to improve the performance of the three phases, respectively. Finally, the proposed algorithm is validated on 18 publicly available real-world datasets in comparison with nine FS algorithms. Experimental results show that the proposed algorithm can obtain a good feature subset with the lowest computational cost.

A structure noise-aware tensor dictionary learning method for high-dimensional data clustering

With the development of data acquisition technology, high-dimensional data clustering is an important yet challenging task in data mining. Despite advances achieved by current clustering methods, they can be further improved. First, many of them usually unfold the high-dimensional data into a large matrix, consequently resulting in destroying the intrinsic structural property. Second, some methods assume that the noise in the dataset conforms to a predefined distribution (e.g., the Gaussian or Laplacian distribution), which violates real-world applications and eventually decreases the clustering performance. To address these issues, in this paper, we propose a novel tensor dictionary learning method for clustering high-dimensional data with the coexistence of structure noise. We adopt tensors, the natural and powerful tools for the generalizations of vectors and matrices, to characterize high-dimensional data. Meanwhile, to depict the noise accurately, we decompose the observed data into clean data, structure noise, and Gaussian noise. Furthermore, we use low-rank tensor modeling to characterize the inherent correlations of clean data and adopt tensor dictionary learning to adaptively and accurately describe the structure noise instead of using the predefined distribution. We design the proximal alternating minimization algorithm to solve the proposed model with the theoretical convergence guarantee. Experimental results on both simulated and real datasets show that the proposed method outperforms the compared methods for high-dimensional data clustering.

A novel minorization–maximization framework for simultaneous feature selection and clustering of high-dimensional count data

A novel minorization–maximization framework for simultaneous feature selection and clustering of high-dimensional count data

Nanoscale defect evaluation framework combining real-time transmission electron microscopy and integrated machine learning-particle filter estimation

Observation of dynamic processes by transmission electron microscopy (TEM) is an attractive technique to experimentally analyze materials’ nanoscale phenomena and understand the microstructure-properties relationships in nanoscale. Even if spatial and temporal resolutions of real-time TEM increase significantly, it is still difficult to say that the researchers quantitatively evaluate the dynamic behavior of defects. Images in TEM video are a two-dimensional projection of three-dimensional space phenomena, thus missing information must be existed that makes image’s uniquely accurate interpretation challenging. Therefore, even though they are still a clustering high-dimensional data and can be compressed to two-dimensional, conventional statistical methods for analyzing images may not be powerful enough to track nanoscale behavior by removing various artifacts associated with experiment; and automated and unbiased processing tools for such big-data are becoming mission-critical to discover knowledge about unforeseen behavior. We have developed a method to quantitative image analysis framework to resolve these problems, in which machine learning and particle filter estimation are uniquely combined. The quantitative and automated measurement of the dislocation velocity in an Fe-31Mn-3Al-3Si autunitic steel subjected to the tensile deformation was performed to validate the framework, and an intermittent motion of the dislocations was quantitatively analyzed. The framework is successfully classifying, identifying and tracking nanoscale objects; these are not able to be accurately implemented by the conventional mean-path based analysis.

A Clustering Algorithm for Tumor Gene Data Based on Improved DPC Algorithm

Cluster analysis is a principal approach to discover unknown tumor subtypes. Innovative and effective cluster analysis methods are of great significance for tumor diagnosis and malignant tumor treatment. Existing studies on the cluster analysis of tumor gene data generally have defects in aspects such as unsatisfactory performance in clustering high-dimensional and high-noise data, and insufficient accuracy in selecting cluster centers. To overcome these defects, this paper performed cluster analysis on tumor gene data based on an improved Density peaks clustering (DPC) algorithm. At first, this paper elaborated on the composition and storage format of tumor tissue samples used in the experiment, gave the tumor gene expression profile data in the matrix format, and introduced the preprocessing process of gene expression profile data. Then, this paper carried out feature selection of tumor gene expression profile data. At last, this paper innovatively divided the target gene density into two parts of K-nearest neighbor local density and neighborhood density, thereby completing the improvement of conventional DPC algorithm and expanding its application scenarios. Combining with experiment, the clustering results of the algorithm before and after introducing the idea of Approximate Nearest Neighbor (ANN) were given, which had verified the effectiveness of the algorithm proposed in this paper.

Projected fuzzy C-means with probabilistic neighbors

In recent years, graph optimization dimensionality reduction methods have become a research hotspot in machine learning. The main challenge of these methods is how to choose proper neighbors for graph construction. For high-dimensional data clustering tasks, most methods often conduct a dimensionality reduction method at first and then perform a clustering method in sequence. However, such a sequential strategy may not be optimal because the reduced data obtained in the first stage may not be suitable for clustering. In this article, a novel method called Projected Fuzzy c-means with Probabilistic Neighbors(PFCM), which unifies graph optimization and Fuzzy c-means, is proposed. Our model projects the data into an optimal subspace at first and then learns the sparse weights matrix by considering probabilistic neighbors and membership matrix together on the projected data. The above two steps run iteratively until the algorithm converges. Especially, L0-norm constraints are employed on the weights matrix to avoid the obstacles caused by outliers. An optimization procedure is designed to solve the proposed model effectively. We conducted numerous experiments on eight benchmark data sets. The experimental results show that the performance of the proposed method is better than some available dimensionality reduction algorithms for clustering tasks.

The [formula omitted]-sparse LSR for subspace clustering via 0-1 integer programming

Subspace clustering is a powerful technology for clustering high-dimensional data. Least squares regression (LSR) is one of the classical subspace clustering models because of the explicit solution and grouping effect that tends to cluster highly correlated data together. However, LSR lacks sparsity, which may lead to its sensitivity to noise and dimension of data. Since the ℓ0-norm constrained optimization is difficult to solve, we introduce a new method based on the 0-1 integer programming, named k-sparse LSR (KSLSR), to directly solve the optimization of the LSR constrained by an ℓ0-norm-based constraint without any relaxation, where k represents the maximum number of non-zero elements of the representation coefficient. With the help of the 0-1 integer constraint, the ℓ0-norm-based constraint is equivalent to two continuous constraints. Thus, the obtained KSLSR method with two continuous constraints is equivalent to the ℓ0-norm constrained optimization and can be effectively solved by the alternating direction multiplier method (ADMM). In addition, the new method is proven to provide a solution with both k-sparsity and the grouping effect, which makes it robust to the noise and dimension of data. Extensive experiments are conducted to demonstrate the effectiveness and superiority of the proposed method.

An Ensemble Clustering Approach (Consensus Clustering) for High-Dimensional Data

Due to the plurality of irrelevant attributes, sparse distribution, and complicated calculations in high-dimensional data, traditional clustering algorithms, such as K-means, do not perform well on high-dimensional data. To address the clustering problem of high-dimensional data, this paper studies an integrated clustering method for high-dimensional data. A method of subspace division based on minimum redundancy is proposed to solve the problem of subspace division of high-dimensional data; subspace division is improved by using the K-means algorithm. Additionally, this method uses mutual information between the characteristic variables of the data to replace the calculation in the K-means algorithm. The distance between the characteristic variables of the data is used to divide the data into subspaces according to the mutual information values between the characteristic variables of the data. To achieve high clustering accuracy and diversity based on clustering requirements, this paper uses a genetic algorithm as the consistency integration function. The fitness function is designed according to the clustering fusion target, and the selection operator is designed according to the maximum number of overlapping elements in the base clustering. The experimental results show that the clustering algorithm proposed in this paper outperforms other methods on most datasets and is an effective clustering integration algorithm. The proposed clustering algorithm is compared with other commonly used clustering fusion algorithms on datasets to prove the advantages of the proposed algorithm.

Robust landmark graph-based clustering for high-dimensional data

High-dimensional data has attracted much attention because it contains more comprehensive information about samples. How to cluster these high-dimensional data has become a crucial topic in unsupervised learning. Existing clustering methods often show limited applicability due to their high computational complexity and low anti-noise ability. To address this issue, we propose a novel robust landmark graph-based clustering algorithm for high-dimensional data (RLGCH), which inherits the advantages of both k-means++ and graph-based clustering by using the results of k-means++ as pseudo labels for landmark graph-based clustering. In particular, RLGCH can achieve more reasonable clustering effectiveness than methods that just operate in the low-dimensional space or the original space since it performs k-means++ in the low-dimensional space and landmark graph-based spectral clustering in the original feature space. To avoid post-processing after optimization, the embedded factor matrix is constrained as an indicator matrix rather than a simple nonnegative matrix. To enhance the clustering robustness, the L2,1-norm is adopted to minimize the error of results between k-means++ and landmark graph-based clustering. To solve the model of RLGCH, we established a novel efficient optimization strategy to obtain all sample categories directly. Combining our clustering model and optimization strategy, the computational complexity is reduced to linear and insensitive to data dimensions. Extensive experiments on seven real-world datasets and sixteen noisy datasets show that compared with other state-of-the-art methods, RLGCH can improve the clustering efficiency and robustness greatly while guaranteeing comparable or even better clustering effectiveness.

Model-based clustering of high-dimensional longitudinal data via regularization.

We propose a model-based clustering method for high-dimensional longitudinal data via regularization in this paper. This study was motivated by the Trial of Activity in Adolescent Girls (TAAG), which aimed to examine multilevel factors related to the change of physical activity by following up a cohort of 783 girls over 10 years from adolescence to early adulthood. Our goal is to identify the intrinsic grouping of subjects with similar patterns of physical activity trajectories and the most relevant predictors within each group. The previous analyses conducted clustering and variable selection in two steps, while our new method can perform the tasks simultaneously. Within each cluster, a linear mixed-effects model (LMM) is fitted with a doubly penalized likelihood to induce sparsity for parameter estimation and effect selection. The large-sample joint properties are established, allowing the dimensions of both fixed and random effects to increase at an exponential rate of the sample size, with a general class of penalty functions. Assuming subjects are drawn from a Gaussian mixture distribution, model effects and cluster labels are estimated via a coordinate descent algorithm nested inside the Expectation-Maximization (EM) algorithm. Bayesian Information Criterion (BIC) is used to determine the optimal number of clusters and the values of tuning parameters. Our numerical studies show that the new method has satisfactory performance and is able to accommodate complex data with multilevel and/or longitudinaleffects.

Clustering high-dimensional data via feature selection.

High-dimensional clustering analysis is a challenging problem in statistics and machine learning, with broad applications such as the analysis of microarray data and RNA-seq data. In this paper, we propose a new clustering procedure called spectral clustering with feature selection (SC-FS), where we first obtain an initial estimate of labels via spectral clustering, then select a small fraction of features with the largest R-squared with these labels, that is, the proportion of variation explained by group labels, and conduct clustering again using selected features. Under mild conditions, we prove that the proposed method identifies all informative features with high probability and achieves the minimax optimal clustering error rate for the sparse Gaussian mixture model. Applications of SC-FS to four real-world datasets demonstrate its usefulness in clustering high-dimensionaldata.

Semi-supervised Latent Block Model with pairwise constraints

Co-clustering aims at simultaneously partitioning both dimensions of a data matrix. It has demonstrated better performances than one-sided clustering for high-dimensional data. The Latent Block Model (LBM) is a probabilistic model for co-clustering based on mixture models that has proven useful for a broad class of data. In this paper, we propose to leverage prior knowledge in the form of pairwise semi-supervision in both row and column space to improve the clustering performances of the algorithms derived from this model. We present a general probabilistic framework for incorporating must link and cannot link relationships in the LBM based on Hidden Markov Random Fields. We instantiate this framework on a model for count data and present two inference algorithms based on Variational and Classification EM. Extensive experiments on simulated data and on real-world attributed networks confirm the interest of our approach and demonstrate the effectiveness of our algorithms.

Investigation of morphotypes of the knee using cluster analysis

BackgroundObjects that manifest in several characteristic shapes, or morphotypes, are typically caused by some hidden variable. For example, the gender of a person influences the width of their pelvis. This is important when reconstructing natural shapes, e.g., in knee implant design. The aim of this study was to identify such morphotypes. MethodsThis work investigated the shapes of roughly 1000 knee joints acquired from computed tomography, including the distal femur and proximal tibia. Two comprehensive feature sets were utilized to describe the bone shapes, one based on morphological measurements and the other on statistical shape model (SSM) weights. We normalized the data by size and performed a cluster analysis with different algorithms, namely k-means and high dimensional data clustering. The clusters were evaluated using several metrics. ResultsThe data showed a low tendency to form clusters. Only one of 12 experiments slightly exceeded the thresholds for actual clusters suggested by the literature. k-Means outperformed high dimensional data clustering in all cases. ConclusionAfter anisotropic normalization by size, which removes size and aspect ratio related differences, the data exhibited no morphotypes. This showed that there are no relevant hidden variables, e.g., gender, body type or ethnicity, which influence the shape of the knee joint. Instead, knee shape is highly individual. Investigating the three-dimensional shape, variations occur for a wide range of different shape parameters, not just for anterior–posterior and mediolateral size.