Clustering High-dimensional Data Research Articles

A Novel Hierarchical High-Dimensional Unsupervised Active Learning Method

This paper processes a novel hierarchical high-dimensional clustering algorithm based on the Active Learning Method (ALM), which is a fuzzy-learning algorithm. The hierarchical part of the algorithm is composed of two phases: divisible and agglomerative. The divisible phase, a zooming-in-process, searches for sub-clusters in already-found clusters hierarchically. At each level of the hierarchy, the clusters are found by an ensemble clustering method based on the density of data. This part of the algorithm blurs each data point as multiple one-dimensional fuzzy membership functions called ink-drop patterns; then, it accumulates the ink-drop patterns of all data points on every dimension separately. Next, it performs one-dimensional density partitioning to produce an ensemble of clustering solutions; after that, combining the results is done based on a novel consensus method with the aid of prime numbers. An agglomerative phase is a bottom-up approach that merges clusters based on a novel distance metric, named K2\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$${K}^{2}$$\\end{document}-nearest neighbor. The algorithm is named as the Hierarchical High-Dimensional Unsupervised Active Learning Method (HiDUALM) and is explained in more detail throughout this paper. Although the classical clustering methods are not suitable for high-dimensional data clustering, the proposed method solves the problems related to speed and memory using ensemble learning, while due to its hierarchy and the use of different distance criteria, different levels of the cluster provide the clause. Experiments on synthetic and real-world datasets are presented to show the effectiveness of the proposed-clustering algorithm.

Clustering statistical method of high dimensional sparse data based on fuzzy data

Abstract Developing effective clustering and statistical methods for high-dimensional sparse data presents unique challenges compared to traditional low-dimensional data. To address this, a novel approach is proposed, leveraging fuzzy data principles to enhance the clustering and statistical performance of high-dimensional sparse datasets. The method builds upon the fuzzy C-means clustering algorithm, introducing key modifications for better suitability to high-dimensional sparse data. One crucial enhancement involves tackling the local optimization problem by optimizing the initial clustering center, significantly reducing clustering statistical time. Replacing the original Euclidean distance with cosine distance improves the clustering and statistical performance of high-dimensional sparse data. Experimental results have shown that this method has superior clustering statistical performance when the data dimensions are different. When the data dimension is low, and the blocking ratio is 10%, the clustering statistical effect is optimal. When the data dimension is high, and the blocking ratio is 40%, the clustering statistical effect is optimal. This method has higher hit rates and clustering statistical efficiency at different sparsity levels.

Projected fuzzy [formula omitted]-means clustering algorithm with instance penalty

At present, high-dimensional data clustering has become a vital research field in machine learning. Traditional clustering algorithms cannot perform well on high-dimensional data, where the clustering task is usually divided into two stages: dimensionality reduction first and clustering later. In general, the existing high-dimensional clustering methods usually have the following shortcomings: (1) the two-stage strategy splits the connection between clustering and dimensionality reduction; (2) these algorithms do not consider the impact of anomalous instances in high-dimensional data on clustering performance. Therefore, to address these problems, a projected fuzzy c-means clustering algorithm with instance penalty (PCIP) is proposed. Firstly, we construct an instance penalty matrix and assign an instance penalty coefficient to each sample. Secondly, a model for clustering high-dimensional data is constructed by integrating fuzzy c-means clustering (FCM) and principal component analysis (PCA). The proposed model can perform dimensionality reduction and clustering simultaneously. In addition, the time complexity of the proposed algorithm is linearly related to the number of samples n, which can efficiently deal with large data sets. The proposed PCIP algorithm is verified by experiments using clustering accuracy and normalized mutual information (NMI) as evaluation metrics. The experimental results on 10 image datasets show that the average accuracy and average NMI of the PCIP algorithm are improved by 0.0375 and 0.0275, respectively, compared to the second-ranked algorithm.

Efficient Feature Clustering for High-Dimensional Datasets: A Non-Parametric Approach

Clustering methods have become widely popular in the field of data analysis, as they enable the grouping of similar data points. However, the challenge of clustering high-dimensional data remains a significant obstacle due to the ”curse of dimensionality.” Traditional methods like Principal Component Analysis (PCA) and hierarchy-based feature agglomeration have been used to overcome this challenge by reducing the dimensionality of the dataset. However, there is currently no efficient method for clustering features of a high-dimensional dataset. To address this issue, we present a non-parametric clustering method in this paper, specifically designed for identifying features in a high-dimensional dataset. Our proposed feature clustering algorithm utilizes a partitioning algorithm that creates clusters of features. Through experimental results using publicly available datasets, we demonstrate the effectiveness of our method. Our approach offers a valuable tool for researchers and practitioners to better understand and analyze high-dimensional data.

IFKMHC: Implicit Fuzzy K-Means Model for High-Dimensional Data Clustering.

The graph-information-based fuzzy clustering has shown promising results in various datasets. However, its performance is hindered when dealing with high-dimensional data due to challenges related to redundant information and sensitivity to the similarity matrix design. To address these limitations, this article proposes an implicit fuzzy k-means (FKMs) model that enhances graph-based fuzzy clustering for high-dimensional data. Instead of explicitly designing a similarity matrix, our approach leverages the fuzzy partition result obtained from the implicit FKMs model to generate an effective similarity matrix. We employ a projection-based technique to handle redundant information, eliminating the need for specific feature extraction methods. By formulating the fuzzy clustering model solely based on the similarity matrix derived from the membership matrix, we mitigate issues, such as dependence on initial values and random fluctuations in clustering results. This innovative approach significantly improves the competitiveness of graph-enhanced fuzzy clustering for high-dimensional data. We present an efficient iterative optimization algorithm for our model and demonstrate its effectiveness through theoretical analysis and experimental comparisons with other state-of-the-art methods, showcasing its superior performance.

High-Dimensional Data Analysis Using Parameter Free Algorithm Data Point Positioning Analysis

Clustering is an effective statistical data analysis technique; it has several applications, including data mining, pattern recognition, image analysis, bioinformatics, and machine learning. Clustering helps to partition data into groups of objects with distinct characteristics. Most of the methods for clustering use manually selected parameters to find the clusters from the dataset. Consequently, it can be very challenging and time-consuming to extract the optimal parameters for clustering a dataset. Moreover, some clustering methods are inadequate for locating clusters in high-dimensional data. To address these concerns systematically, this paper introduces a novel selection-free clustering technique named data point positioning analysis (DPPA). The proposed method is straightforward since it calculates 1-NN and Max-NN by analyzing the data point placements without the requirement of an initial manual parameter assignment. This method is validated using two well-known publicly available datasets used in several clustering algorithms. To compare the performance of the proposed method, this study also investigated four popular clustering algorithms (DBSCAN, affinity propagation, Mean Shift, and K-means), where the proposed method provides higher performance in finding the cluster without using any manually selected parameters. The experimental finding demonstrated that the proposed DPPA algorithm is less time-consuming compared to the existing traditional methods and achieves higher performance without using any manually selected parameters.

Tuning-free sparse clustering via alternating hard-thresholding

Model-based clustering is a commonly-used technique to partition heterogeneous data into homogeneous groups. When the analysis is to be conducted with a large number of features, analysts face simultaneous challenges in model interpretability, clustering accuracy, and computational efficiency. Several Bayesian and penalization methods have been proposed to select important features for model-based clustering. However, the performance of those methods relies on a careful algorithmic tuning, which can be time-consuming for high-dimensional cases. In this paper, we propose a new sparse clustering method based on alternating hard-thresholding. The new method is conceptually simple and tuning-free. With a user-specified sparsity level, it efficiently detects a set of key features by eliminating a large number of features that are less useful for clustering. Based on the selected key features, one can readily obtain an effective clustering of the original high-dimensional data under a general sparse covariance structure. Under mild conditions, we show that the new method leads to clusters with a misclassification rate consistent to the optimal rate as if the underlying true model were used. The promising performance of the new method is supported by both simulated and real data examples.

Robust and compact maximum margin clustering for high-dimensional data

In the field of machine learning, clustering has become an increasingly popular research topic due to its critical importance. Many clustering algorithms have been proposed utilizing a variety of approaches. This study focuses on clustering of high-dimensional data using the maximum margin clustering approach. In this paper, two methods are introduced: The first method employs the classical maximum margin clustering approach, which separates data into two clusters with the greatest margin between them. The second method takes cluster compactness into account and searches for two parallel hyperplanes that best fit to the cluster samples while also being as far apart from each other as possible. Additionally, robust variants of these clustering methods are introduced to handle outliers and noise within the data samples. The stochastic gradient algorithm is used to solve the resulting optimization problems, enabling all proposed clustering methods to scale well with large-scale data. Experimental results demonstrate that the proposed methods are more effective than existing maximum margin clustering methods, particularly in high-dimensional clustering problems, highlighting the efficacy of the proposed methods.

Clustering High-Dimensional Noisy Categorical Data

Clustering is a widely used unsupervised learning technique that groups data into homogeneous clusters. However, when dealing with real-world data that contain categorical values, existing algorithms can be computationally costly in high dimensions and can struggle with noisy data that has missing values. Furthermore, except for one algorithm, no others provide theoretical guarantees of clustering accuracy. In this article, we propose a general categorical data encoding method and a computationally efficient spectral-based algorithm to cluster high-dimensional noisy categorical data (nominal or ordinal). Under a statistical model for data on m attributes from n subjects in r clusters with missing probability ϵ, we show that our algorithm exactly recovers the true clusters with high probability when m n ( 1 − ϵ ) ≥ C M r 2 log 3 M , with M = max ( n , m ) and a fixed constant C. In addition, we show that m n ( 1 − ϵ ) 2 ≥ r δ / 2 with 0 < δ < 1 is necessary for any algorithm to succeed with probability at least ( 1 + δ ) / 2 . In cases where m = n and r are fixed, the sufficient condition matches with the necessary condition up to a polylog ( n ) factor. In numerical studies our algorithm outperforms several existing algorithms in both clustering accuracy and computational efficiency. Supplementary materials for this article are available online.

Improved Selective Deep-Learning-Based Clustering Ensemble

Clustering ensemble integrates multiple base clustering results to improve the stability and robustness of the single clustering method. It consists of two principal steps: a generation step, which is about the creation of base clusterings, and a consensus function, which is the integration of all clusterings obtained in the generation step. However, most of the existing base clustering algorithms used in the generation step are shallow clustering algorithms such as k-means. These shallow clustering algorithms do not work well or even fail when dealing with large-scale, high-dimensional unstructured data. The emergence of deep clustering algorithms provides a solution to address this challenge. Deep clustering combines the unsupervised commonality of deep representation learning to address complex high-dimensional data clustering, which has achieved excellent performance in many fields. In light of this, we introduce deep clustering into clustering ensemble and propose an improved selective deep-learning-based clustering ensemble algorithm (ISDCE). ISDCE exploits the deep clustering algorithm with different initialization parameters to generate multiple diverse base clusterings. Next, ISDCE constructs ensemble quality and diversity evaluation metrics of base clusterings to select higher-quality and rich-diversity candidate base clusterings. Finally, a weighted graph partition consensus function is utilized to aggregate the candidate base clusterings to obtain a consensus clustering result. Extensive experimental results on various types of datasets demonstrate that ISDCE performs significantly better than existing clustering ensemble approaches.

Mixtures of factor analysers with censored responses and external covariates: An application to educational data from Peru.

Analysing data from educational tests allows governments to make decisions for improving the quality of life of individuals in a society. One of the key responsibilities of statisticians is to develop models that provide decision-makers with pertinent information about the latent process that educational tests seek to represent. Mixtures of factor analysers (MtFA) have emerged as a powerful device for model-based clustering and classification of high-dimensional data containing one or several groups of observations with fatter tails or anomalous outliers. This paper considers an extension of MtFA for robust clustering of censored data, referred to as the MtFAC model, by incorporating external covariates. The enhanced flexibility of including covariates in MtFAC enables cluster-specific multivariate regression analysis of dependent variables with censored responses arising from upper and/or lower detection limits of experimental equipment. An alternating expectation conditional maximization (AECM) algorithm is developed for maximum likelihood estimation of the proposed model. Two simulation experiments are conducted to examine the effectiveness of the techniques presented. Furthermore, the proposed methodology is applied to Peruvian data from the 2007 Early Grade Reading Assessment, and the results obtained from the analysis provide new insights regarding the reading skills of Peruvian students.

Statistical method for clustering high-dimensional data based on fuzzy mathematical modeling

Abstract In this paper, firstly, the fuzzy mathematical model and its types and algorithms of fuzzy numbers are studied. Then, the concept of fuzzy mathematics is added to cluster analysis, and fuzzy cluster analysis is carried out on samples or objects after data standardization and the construction of a similarity matrix. Then, the fuzzy C-mean FCM algorithm is proposed, and the FCM algorithm is improved by introducing the point density of data objects, the maximum minimum distance method of point density sampling, and the reduction of computation to improve the statistical function of clustering high-dimensional data. Finally, the algorithm of this paper is compared with other algorithms by AC, PR, RE, convergence speed, running time, DBI index and other indexes in order to analyze the advantages of the FCM algorithm in high dimensional data clustering statistics. The results show that the F value of the FCM algorithm in the Zoo dataset reaches 0.976, which improves 61% compared to FKM and nearly 12.6% compared to the IIFKM0 algorithm, with the best clustering effect. The DBI index of the FCM algorithm is the lowest in the four datasets, which is basically stabilized in the range of 0.5-0.6. This is better than other algorithms.

Incomplete multiview subspace clustering based on multiple kernel low-redundant representation learning

Subspace clustering is a widely used technique for clustering high-dimensional data. However, its effectiveness is limited in the context of incomplete multiview clustering, where intact subspaces cannot be obtained due to missing instances. To address this issue, we present a novel approach for incomplete multiview subspace clustering based on multiple kernel completion, low-redundant representation learning, and weighted tensor low-rank constraint. First, a carefully designed kernel completion scheme is employed to obtain intact kernels, from which the complete low-redundant representations are learned to obtain intact and compact subspaces. Second, unlike the traditional pairwise subspace fusion, we propose to fuse the multiview subspaces with a weighted tensor low-rank constraint, which not only explores higher-order relationships among views but also assigns appropriate weights to each view. Finally, we propose a unified model that jointly learns low-redundant representations, view-specific subspaces, and their low-rank tensor structure. Extensive experiments conducted on four publicly available datasets demonstrate the effectiveness of the proposed method.

Adaptive local Principal Component Analysis improves the clustering of high-dimensional data

In local Principal Component Analysis (PCA), a distribution is approximated by multiple units, each representing a local region by a hyper-ellipsoid obtained through PCA. We present an extension for local PCA which adaptively adjusts both the learning rate of each unit and the potential function which guides the competition between the local units. Our local PCA method is an online neural network method where unit centers and shapes are modified after the presentation of each data point. For several benchmark distributions, we demonstrate that our method improves the overall quality of clustering, especially for high-dimensional distributions where many conventional methods do not perform satisfactorily. Our online method is also well suited for the processing of streaming data: The two adaptive mechanisms lead to a quick reorganization of the clustering when the underlying distribution changes.

Hypergraph-Clustering Method Based on an Improved Apriori Algorithm

With the complexity and variability of data structures and dimensions, traditional clustering algorithms face various challenges. The integration of network science and clustering has become a popular field of exploration. One of the main challenges is how to handle large-scale and complex high-dimensional data effectively. Hypergraphs can accurately represent multidimensional heterogeneous data, making them important for improving clustering performance. In this paper, we propose a hypergraph-clustering method dubbed the “high-dimensional data clustering method” based on hypergraph partitioning using an improved Apriori algorithm (HDHPA). First, the method constructs a hypergraph based on the improved Apriori association rule algorithm, where frequent itemsets existing in high-dimensional data are treated as hyperedges. Then, different frequent itemsets are mined in parallel to obtain hyperedges with corresponding ranks, avoiding the generation of redundant rules and improving mining efficiency. Next, we use the dense subgraph partition (DSP) algorithm to divide the hypergraph into multiple subclusters. Finally, we merge the subclusters through dense sub-hypergraphs to obtain the clustering results. The advantage of this method lies in its use of the hypergraph model to discretize the association between data in space, which further enhances the effectiveness and accuracy of clustering. We comprehensively compare the proposed HDHPA method with several advanced hypergraph-clustering methods using seven different types of high-dimensional datasets and then compare their running times. The results show that the clustering evaluation index values of the HDHPA method are generally superior to all other methods. The maximum ARI value can reach 0.834, an increase of 42%, and the average running time is lower than other methods. All in all, HDHPA exhibits an excellent comparable performance on multiple real networks. The research results of this paper provide an effective solution for processing and analyzing large-scale network datasets and are also conducive to broadening the application range of clustering techniques.

Kernel Block Diagonal Representation Subspace Clustering with Similarity Preservation

Subspace clustering methods based on the low-rank and sparse model are effective strategies for high-dimensional data clustering. However, most existing low-rank and sparse methods with self-expression can only deal with linear structure data effectively, but they cannot handle data with complex nonlinear structure well. Although kernel subspace clustering methods can efficiently deal with nonlinear structure data, some similarity information between samples may be lost when the original data are reconstructed in the kernel space. Moreover, these kernel subspace clustering methods may not obtain an affinity matrix with an optimal block diagonal structure. In this paper, we propose a novel subspace clustering method termed kernel block diagonal representation subspace clustering with similarity preservation (KBDSP). KBDSP contains three contributions: (1) an affinity matrix with block diagonal structure is generated by introducing a block diagonal representation term; (2) a similarity-preserving regularizer is constructed and embedded into our model by minimizing the discrepancy between inner products of original data and inner products of reconstructed data in the kernel space, which better preserve the similarity information between original data; (3) the KBDSP model is proposed by integrating the block diagonal representation term and similarity-preserving regularizer into the kernel self-expressing frame. The optimization of our proposed model is solved efficiently by utilizing the alternating direction method of multipliers (ADMM). Experimental results on nine datasets demonstrate the effectiveness of the proposed method.

Sparse kernel k-means for high-dimensional data

The kernel k-means method usually loses its power when clustering high-dimensional data, due to a large number of irrelevant features. We propose a novel sparse kernel k-means clustering (SKKM) to extend the advantages of kernel k-means to the high-dimensional cases. We assign each feature a 0-1 indicator and optimize an equivalent kernel k-means loss function while penalizing the sum of the indicators. An alternating minimization algorithm is proposed to estimate both the class labels and the feature indicators. We prove the consistency of both clustering and feature selection of the proposed method. In addition, we apply the proposed framework to the normalized cut. In the numerical experiments, we demonstrate that the proposed method provides better/comparable performance compared to the existing high-dimensional clustering methods.

Retracted: An Ensemble Clustering Approach (Consensus Clustering) for High-Dimensional Data

Retracted: An Ensemble Clustering Approach (Consensus Clustering) for High-Dimensional Data

UEQMS: UMAP Embedded Quick Mean Shift Algorithm for High Dimensional Clustering

The mean shift algorithm is a simple yet very effective clustering method widely used for image and video segmentation as well as other exploratory data analysis applications. Recently, a new algorithm called MeanShift++ (MS++) for low-dimensional clustering was proposed with a speedup of 4000 times over the vanilla mean shift. In this work, starting with a first-of-its-kind theoretical analysis of MS++, we extend its reach to high-dimensional data clustering by integrating the Uniform Manifold Approximation and Projection (UMAP) based dimensionality reduction in the same framework. Analytically, we show that MS++ can indeed converge to a non-critical point. Subsequently, we suggest modifications to MS++ to improve its convergence characteristics. In addition, we propose a way to further speed up MS++ by avoiding the execution of the MS++ iterations for every data point. By incorporating UMAP with modified MS++, we design a faster algorithm, named UMAP embedded quick mean shift (UEQMS), for partitioning data with a relatively large number of recorded features. Through extensive experiments, we showcase the efficacy of UEQMS over other state-of-the-art algorithms in terms of accuracy and runtime.

A Proposed Clustering Algorithm for Efficient Clustering of High-Dimensional Data

To partition transaction data values, clustering algorithms are used. To analyse the relationships between transactions, similarity measures are utilized. Similarity models based on vectors perform well with low-dimensional data. High-dimensional data values are clustered using subspace clustering techniques. Clustering high-dimensional data is difficult due to the curse of dimensionality. Projective clustering seeks out projected clusters in subsets of a data space's dimensions. In high-dimensional data space, a probability model represents predicted clusters. A model-based fuzzy projection clustering method to find clusters with overlapping boundaries in different projection subspaces. The system employs the Model Based Projective Clustering (MPC) method. To cluster high-dimensional data, projective clustering algorithms are used. A subspace clustering technique is the model-based projective clustering algorithm. Similarity analysis use non-axis-subspaces. Anomaly transactions are segmented using projected clusters. The suggested system is intended to cluster objects in high-dimensional spaces. The similarity analysis includes non-access subspaces. The clustering procedure validates anomaly data values with similarity. The subspace selection procedure has been optimized. A subspace clustering approach is the model-based projective clustering algorithm. Similarity analysis use non-axis-subspaces. Anomaly transactions are segmented using projected clusters. The suggested system is intended to cluster objects in high-dimensional spaces. The similarity analysis includes non-access subspaces. The clustering procedure validates anomaly data values with similarity. The subspace selection procedure has been improved.