Clustering High-dimensional Data Research Articles

Efficient Feature Clustering for High-Dimensional Datasets: A Non-Parametric Approach

Clustering methods have become widely popular in the field of data analysis, as they enable the grouping of similar data points. However, the challenge of clustering high-dimensional data remains a significant obstacle due to the ”curse of dimensionality.” Traditional methods like Principal Component Analysis (PCA) and hierarchy-based feature agglomeration have been used to overcome this challenge by reducing the dimensionality of the dataset. However, there is currently no efficient method for clustering features of a high-dimensional dataset. To address this issue, we present a non-parametric clustering method in this paper, specifically designed for identifying features in a high-dimensional dataset. Our proposed feature clustering algorithm utilizes a partitioning algorithm that creates clusters of features. Through experimental results using publicly available datasets, we demonstrate the effectiveness of our method. Our approach offers a valuable tool for researchers and practitioners to better understand and analyze high-dimensional data.

IFKMHC: Implicit Fuzzy K-Means Model for High-Dimensional Data Clustering.

The graph-information-based fuzzy clustering has shown promising results in various datasets. However, its performance is hindered when dealing with high-dimensional data due to challenges related to redundant information and sensitivity to the similarity matrix design. To address these limitations, this article proposes an implicit fuzzy k-means (FKMs) model that enhances graph-based fuzzy clustering for high-dimensional data. Instead of explicitly designing a similarity matrix, our approach leverages the fuzzy partition result obtained from the implicit FKMs model to generate an effective similarity matrix. We employ a projection-based technique to handle redundant information, eliminating the need for specific feature extraction methods. By formulating the fuzzy clustering model solely based on the similarity matrix derived from the membership matrix, we mitigate issues, such as dependence on initial values and random fluctuations in clustering results. This innovative approach significantly improves the competitiveness of graph-enhanced fuzzy clustering for high-dimensional data. We present an efficient iterative optimization algorithm for our model and demonstrate its effectiveness through theoretical analysis and experimental comparisons with other state-of-the-art methods, showcasing its superior performance.

High-Dimensional Data Analysis Using Parameter Free Algorithm Data Point Positioning Analysis

Clustering is an effective statistical data analysis technique; it has several applications, including data mining, pattern recognition, image analysis, bioinformatics, and machine learning. Clustering helps to partition data into groups of objects with distinct characteristics. Most of the methods for clustering use manually selected parameters to find the clusters from the dataset. Consequently, it can be very challenging and time-consuming to extract the optimal parameters for clustering a dataset. Moreover, some clustering methods are inadequate for locating clusters in high-dimensional data. To address these concerns systematically, this paper introduces a novel selection-free clustering technique named data point positioning analysis (DPPA). The proposed method is straightforward since it calculates 1-NN and Max-NN by analyzing the data point placements without the requirement of an initial manual parameter assignment. This method is validated using two well-known publicly available datasets used in several clustering algorithms. To compare the performance of the proposed method, this study also investigated four popular clustering algorithms (DBSCAN, affinity propagation, Mean Shift, and K-means), where the proposed method provides higher performance in finding the cluster without using any manually selected parameters. The experimental finding demonstrated that the proposed DPPA algorithm is less time-consuming compared to the existing traditional methods and achieves higher performance without using any manually selected parameters.

Tuning-free sparse clustering via alternating hard-thresholding

Model-based clustering is a commonly-used technique to partition heterogeneous data into homogeneous groups. When the analysis is to be conducted with a large number of features, analysts face simultaneous challenges in model interpretability, clustering accuracy, and computational efficiency. Several Bayesian and penalization methods have been proposed to select important features for model-based clustering. However, the performance of those methods relies on a careful algorithmic tuning, which can be time-consuming for high-dimensional cases. In this paper, we propose a new sparse clustering method based on alternating hard-thresholding. The new method is conceptually simple and tuning-free. With a user-specified sparsity level, it efficiently detects a set of key features by eliminating a large number of features that are less useful for clustering. Based on the selected key features, one can readily obtain an effective clustering of the original high-dimensional data under a general sparse covariance structure. Under mild conditions, we show that the new method leads to clusters with a misclassification rate consistent to the optimal rate as if the underlying true model were used. The promising performance of the new method is supported by both simulated and real data examples.

Robust and compact maximum margin clustering for high-dimensional data

In the field of machine learning, clustering has become an increasingly popular research topic due to its critical importance. Many clustering algorithms have been proposed utilizing a variety of approaches. This study focuses on clustering of high-dimensional data using the maximum margin clustering approach. In this paper, two methods are introduced: The first method employs the classical maximum margin clustering approach, which separates data into two clusters with the greatest margin between them. The second method takes cluster compactness into account and searches for two parallel hyperplanes that best fit to the cluster samples while also being as far apart from each other as possible. Additionally, robust variants of these clustering methods are introduced to handle outliers and noise within the data samples. The stochastic gradient algorithm is used to solve the resulting optimization problems, enabling all proposed clustering methods to scale well with large-scale data. Experimental results demonstrate that the proposed methods are more effective than existing maximum margin clustering methods, particularly in high-dimensional clustering problems, highlighting the efficacy of the proposed methods.

Clustering High-Dimensional Noisy Categorical Data

Clustering is a widely used unsupervised learning technique that groups data into homogeneous clusters. However, when dealing with real-world data that contain categorical values, existing algorithms can be computationally costly in high dimensions and can struggle with noisy data that has missing values. Furthermore, except for one algorithm, no others provide theoretical guarantees of clustering accuracy. In this article, we propose a general categorical data encoding method and a computationally efficient spectral-based algorithm to cluster high-dimensional noisy categorical data (nominal or ordinal). Under a statistical model for data on m attributes from n subjects in r clusters with missing probability ϵ, we show that our algorithm exactly recovers the true clusters with high probability when m n ( 1 − ϵ ) ≥ C M r 2 log 3 M , with M = max ( n , m ) and a fixed constant C. In addition, we show that m n ( 1 − ϵ ) 2 ≥ r δ / 2 with 0 < δ < 1 is necessary for any algorithm to succeed with probability at least ( 1 + δ ) / 2 . In cases where m = n and r are fixed, the sufficient condition matches with the necessary condition up to a polylog ( n ) factor. In numerical studies our algorithm outperforms several existing algorithms in both clustering accuracy and computational efficiency. Supplementary materials for this article are available online.

Improved Selective Deep-Learning-Based Clustering Ensemble

Clustering ensemble integrates multiple base clustering results to improve the stability and robustness of the single clustering method. It consists of two principal steps: a generation step, which is about the creation of base clusterings, and a consensus function, which is the integration of all clusterings obtained in the generation step. However, most of the existing base clustering algorithms used in the generation step are shallow clustering algorithms such as k-means. These shallow clustering algorithms do not work well or even fail when dealing with large-scale, high-dimensional unstructured data. The emergence of deep clustering algorithms provides a solution to address this challenge. Deep clustering combines the unsupervised commonality of deep representation learning to address complex high-dimensional data clustering, which has achieved excellent performance in many fields. In light of this, we introduce deep clustering into clustering ensemble and propose an improved selective deep-learning-based clustering ensemble algorithm (ISDCE). ISDCE exploits the deep clustering algorithm with different initialization parameters to generate multiple diverse base clusterings. Next, ISDCE constructs ensemble quality and diversity evaluation metrics of base clusterings to select higher-quality and rich-diversity candidate base clusterings. Finally, a weighted graph partition consensus function is utilized to aggregate the candidate base clusterings to obtain a consensus clustering result. Extensive experimental results on various types of datasets demonstrate that ISDCE performs significantly better than existing clustering ensemble approaches.

Mixtures of factor analysers with censored responses and external covariates: An application to educational data from Peru.

Analysing data from educational tests allows governments to make decisions for improving the quality of life of individuals in a society. One of the key responsibilities of statisticians is to develop models that provide decision-makers with pertinent information about the latent process that educational tests seek to represent. Mixtures of factor analysers (MtFA) have emerged as a powerful device for model-based clustering and classification of high-dimensional data containing one or several groups of observations with fatter tails or anomalous outliers. This paper considers an extension of MtFA for robust clustering of censored data, referred to as the MtFAC model, by incorporating external covariates. The enhanced flexibility of including covariates in MtFAC enables cluster-specific multivariate regression analysis of dependent variables with censored responses arising from upper and/or lower detection limits of experimental equipment. An alternating expectation conditional maximization (AECM) algorithm is developed for maximum likelihood estimation of the proposed model. Two simulation experiments are conducted to examine the effectiveness of the techniques presented. Furthermore, the proposed methodology is applied to Peruvian data from the 2007 Early Grade Reading Assessment, and the results obtained from the analysis provide new insights regarding the reading skills of Peruvian students.

Statistical method for clustering high-dimensional data based on fuzzy mathematical modeling

Abstract In this paper, firstly, the fuzzy mathematical model and its types and algorithms of fuzzy numbers are studied. Then, the concept of fuzzy mathematics is added to cluster analysis, and fuzzy cluster analysis is carried out on samples or objects after data standardization and the construction of a similarity matrix. Then, the fuzzy C-mean FCM algorithm is proposed, and the FCM algorithm is improved by introducing the point density of data objects, the maximum minimum distance method of point density sampling, and the reduction of computation to improve the statistical function of clustering high-dimensional data. Finally, the algorithm of this paper is compared with other algorithms by AC, PR, RE, convergence speed, running time, DBI index and other indexes in order to analyze the advantages of the FCM algorithm in high dimensional data clustering statistics. The results show that the F value of the FCM algorithm in the Zoo dataset reaches 0.976, which improves 61% compared to FKM and nearly 12.6% compared to the IIFKM0 algorithm, with the best clustering effect. The DBI index of the FCM algorithm is the lowest in the four datasets, which is basically stabilized in the range of 0.5-0.6. This is better than other algorithms.

Incomplete multiview subspace clustering based on multiple kernel low-redundant representation learning

Subspace clustering is a widely used technique for clustering high-dimensional data. However, its effectiveness is limited in the context of incomplete multiview clustering, where intact subspaces cannot be obtained due to missing instances. To address this issue, we present a novel approach for incomplete multiview subspace clustering based on multiple kernel completion, low-redundant representation learning, and weighted tensor low-rank constraint. First, a carefully designed kernel completion scheme is employed to obtain intact kernels, from which the complete low-redundant representations are learned to obtain intact and compact subspaces. Second, unlike the traditional pairwise subspace fusion, we propose to fuse the multiview subspaces with a weighted tensor low-rank constraint, which not only explores higher-order relationships among views but also assigns appropriate weights to each view. Finally, we propose a unified model that jointly learns low-redundant representations, view-specific subspaces, and their low-rank tensor structure. Extensive experiments conducted on four publicly available datasets demonstrate the effectiveness of the proposed method.

Adaptive local Principal Component Analysis improves the clustering of high-dimensional data

In local Principal Component Analysis (PCA), a distribution is approximated by multiple units, each representing a local region by a hyper-ellipsoid obtained through PCA. We present an extension for local PCA which adaptively adjusts both the learning rate of each unit and the potential function which guides the competition between the local units. Our local PCA method is an online neural network method where unit centers and shapes are modified after the presentation of each data point. For several benchmark distributions, we demonstrate that our method improves the overall quality of clustering, especially for high-dimensional distributions where many conventional methods do not perform satisfactorily. Our online method is also well suited for the processing of streaming data: The two adaptive mechanisms lead to a quick reorganization of the clustering when the underlying distribution changes.

Hypergraph-Clustering Method Based on an Improved Apriori Algorithm

With the complexity and variability of data structures and dimensions, traditional clustering algorithms face various challenges. The integration of network science and clustering has become a popular field of exploration. One of the main challenges is how to handle large-scale and complex high-dimensional data effectively. Hypergraphs can accurately represent multidimensional heterogeneous data, making them important for improving clustering performance. In this paper, we propose a hypergraph-clustering method dubbed the “high-dimensional data clustering method” based on hypergraph partitioning using an improved Apriori algorithm (HDHPA). First, the method constructs a hypergraph based on the improved Apriori association rule algorithm, where frequent itemsets existing in high-dimensional data are treated as hyperedges. Then, different frequent itemsets are mined in parallel to obtain hyperedges with corresponding ranks, avoiding the generation of redundant rules and improving mining efficiency. Next, we use the dense subgraph partition (DSP) algorithm to divide the hypergraph into multiple subclusters. Finally, we merge the subclusters through dense sub-hypergraphs to obtain the clustering results. The advantage of this method lies in its use of the hypergraph model to discretize the association between data in space, which further enhances the effectiveness and accuracy of clustering. We comprehensively compare the proposed HDHPA method with several advanced hypergraph-clustering methods using seven different types of high-dimensional datasets and then compare their running times. The results show that the clustering evaluation index values of the HDHPA method are generally superior to all other methods. The maximum ARI value can reach 0.834, an increase of 42%, and the average running time is lower than other methods. All in all, HDHPA exhibits an excellent comparable performance on multiple real networks. The research results of this paper provide an effective solution for processing and analyzing large-scale network datasets and are also conducive to broadening the application range of clustering techniques.

Kernel Block Diagonal Representation Subspace Clustering with Similarity Preservation

Subspace clustering methods based on the low-rank and sparse model are effective strategies for high-dimensional data clustering. However, most existing low-rank and sparse methods with self-expression can only deal with linear structure data effectively, but they cannot handle data with complex nonlinear structure well. Although kernel subspace clustering methods can efficiently deal with nonlinear structure data, some similarity information between samples may be lost when the original data are reconstructed in the kernel space. Moreover, these kernel subspace clustering methods may not obtain an affinity matrix with an optimal block diagonal structure. In this paper, we propose a novel subspace clustering method termed kernel block diagonal representation subspace clustering with similarity preservation (KBDSP). KBDSP contains three contributions: (1) an affinity matrix with block diagonal structure is generated by introducing a block diagonal representation term; (2) a similarity-preserving regularizer is constructed and embedded into our model by minimizing the discrepancy between inner products of original data and inner products of reconstructed data in the kernel space, which better preserve the similarity information between original data; (3) the KBDSP model is proposed by integrating the block diagonal representation term and similarity-preserving regularizer into the kernel self-expressing frame. The optimization of our proposed model is solved efficiently by utilizing the alternating direction method of multipliers (ADMM). Experimental results on nine datasets demonstrate the effectiveness of the proposed method.

Sparse kernel k-means for high-dimensional data

The kernel k-means method usually loses its power when clustering high-dimensional data, due to a large number of irrelevant features. We propose a novel sparse kernel k-means clustering (SKKM) to extend the advantages of kernel k-means to the high-dimensional cases. We assign each feature a 0-1 indicator and optimize an equivalent kernel k-means loss function while penalizing the sum of the indicators. An alternating minimization algorithm is proposed to estimate both the class labels and the feature indicators. We prove the consistency of both clustering and feature selection of the proposed method. In addition, we apply the proposed framework to the normalized cut. In the numerical experiments, we demonstrate that the proposed method provides better/comparable performance compared to the existing high-dimensional clustering methods.

Retracted: An Ensemble Clustering Approach (Consensus Clustering) for High-Dimensional Data

Retracted: An Ensemble Clustering Approach (Consensus Clustering) for High-Dimensional Data

UEQMS: UMAP Embedded Quick Mean Shift Algorithm for High Dimensional Clustering

The mean shift algorithm is a simple yet very effective clustering method widely used for image and video segmentation as well as other exploratory data analysis applications. Recently, a new algorithm called MeanShift++ (MS++) for low-dimensional clustering was proposed with a speedup of 4000 times over the vanilla mean shift. In this work, starting with a first-of-its-kind theoretical analysis of MS++, we extend its reach to high-dimensional data clustering by integrating the Uniform Manifold Approximation and Projection (UMAP) based dimensionality reduction in the same framework. Analytically, we show that MS++ can indeed converge to a non-critical point. Subsequently, we suggest modifications to MS++ to improve its convergence characteristics. In addition, we propose a way to further speed up MS++ by avoiding the execution of the MS++ iterations for every data point. By incorporating UMAP with modified MS++, we design a faster algorithm, named UMAP embedded quick mean shift (UEQMS), for partitioning data with a relatively large number of recorded features. Through extensive experiments, we showcase the efficacy of UEQMS over other state-of-the-art algorithms in terms of accuracy and runtime.

A Proposed Clustering Algorithm for Efficient Clustering of High-Dimensional Data

To partition transaction data values, clustering algorithms are used. To analyse the relationships between transactions, similarity measures are utilized. Similarity models based on vectors perform well with low-dimensional data. High-dimensional data values are clustered using subspace clustering techniques. Clustering high-dimensional data is difficult due to the curse of dimensionality. Projective clustering seeks out projected clusters in subsets of a data space's dimensions. In high-dimensional data space, a probability model represents predicted clusters. A model-based fuzzy projection clustering method to find clusters with overlapping boundaries in different projection subspaces. The system employs the Model Based Projective Clustering (MPC) method. To cluster high-dimensional data, projective clustering algorithms are used. A subspace clustering technique is the model-based projective clustering algorithm. Similarity analysis use non-axis-subspaces. Anomaly transactions are segmented using projected clusters. The suggested system is intended to cluster objects in high-dimensional spaces. The similarity analysis includes non-access subspaces. The clustering procedure validates anomaly data values with similarity. The subspace selection procedure has been optimized. A subspace clustering approach is the model-based projective clustering algorithm. Similarity analysis use non-axis-subspaces. Anomaly transactions are segmented using projected clusters. The suggested system is intended to cluster objects in high-dimensional spaces. The similarity analysis includes non-access subspaces. The clustering procedure validates anomaly data values with similarity. The subspace selection procedure has been improved.

Joint feature selection and optimal bipartite graph learning for subspace clustering

Recently, there has been tremendous interest in developing graph-based subspace clustering in high-dimensional data, which does not require a priori knowledge of the number of dimensions and subspaces. The general steps of such algorithms are dictionary representation and spectral clustering. Traditional methods use the dataset itself as a dictionary when performing dictionary representation. There are some limitations that the redundant information present in the dictionary and features may make the constructed graph structure unclear and require post-processing to obtain labels. To address these problems, we propose a novel subspace clustering model that first introduces feature selection to process the input data, randomly selects some samples to construct a dictionary to remove redundant information and learns the optimal bipartite graph with K-connected components under the constraint of the (normalized) Laplacian rank. Finally, the labels are obtained directly from the graphs. The experimental results on motion segmentation and face recognition datasets demonstrate the superior effectiveness and stability of our algorithm.

Deep Multi-Constraint Soft Clustering Analysis for Single-Cell RNA-Seq Data via Zero-Inflated Autoencoder Embedding.

Clustering cells into subgroups plays a critical role in single cell-based analyses, which facilitates to reveal cell heterogeneity and diversity. Due to the ever-increasing scRNA-seq data and low RNA capture rate, it has become challenging to cluster high-dimensional and sparse scRNA-seq data. In this study, we propose a single-cell Multi-Constraint deep soft K-means Clustering(scMCKC) framework. Based on zero-inflated negative binomial (ZINB) model-based autoencoder, scMCKC constructs a novel cell-level compactness constraint by considering association between similar cell, to emphasize the compactness between clusters. Besides, scMCKC utilizes pairwise constraint encoded by prior information to guide clustering. Meanwhile, a weighted soft K-means algorithm is leveraged to determine the cell populations, which assigns the label based on affinity between data and clustering center. Experiments on eleven scRNA-seq datasets demonstrate that scMCKC is superior to the state-of-the-art methods and notably improves cluster performance. Moreover, we validate the robustness on human kidney dataset, which demonstrates that scMCKC exhibits comprehensively excellent performance on clustering analysis. The ablation study on eleven datasets proves that the novel cell-level compactness constraint is conductive to the clustering results.

HSCFC: High-dimensional streaming data clustering algorithm based on feedback control system

At present, few studies have considered the clustering analysis of high-dimensional data collected by many sensors in a real-time streaming environment. Existing clustering analysis algorithms for high-dimensional data are primarily based on batch processing models, and most of them cannot meet the requirements of incremental high-dimensional data streams, which are extremely common in practical applications. To address the aforementioned problems, this paper focuses on the study of high-dimensional data clustering based on stream processing, and proposes a high-dimensional data stream clustering algorithm based on a feedback control system, which comprises three stages: window principal component analysis, feedback stream clustering, and feedback controller. The classic exponentially weighted attenuation function is used in the window principal component analysis to avoid concept drift in the data stream, and incremental feature extraction executes through the sliding window to improve the iterative efficiency of the data in the window. To minimize the errors caused by variability in projection angles during dimensionality reduction, a feedback stream clustering stage is designed with alternating iterations of window clustering and cluster aggregation. Aiming at the problems caused by manually adjusting the hyperparameters used in high-dimensional data stream clustering, a feedback controller is developed to adjust the hyperparameters in the two other stages by analyzing the clustering results in real time and using a discriminant score to adopt corresponding feedback strategies. The experimental comparisons between the proposed and the traditional algorithms on multiple datasets demonstrate the effectiveness of the former.