Clustering High-dimensional Data Research Articles

Local gap density for clustering high-dimensional data with varying densities

Density-based clustering algorithms are for clustering the data with arbitrary shapes. However, most of these algorithms face difficulties in handling the high-dimensional data with varying densities; especially, they cannot well discover the clusters in sparse regions. In this paper, we define a new type of density, local gap density, in the k-NN graph which works well for high-dimensional data. The local gap density of each point considers not only the number of all points in its nearest neighbor but also the average distance from this point to all points in this nearest neighbor. In this way, the core points in sparse regions in the sense of existing density-based clustering have high densities in our density definition, so they can be easily detected. By the core points, the potential cross-cluster edges in the k-NN graph can be well identified. After deleting these edges, we group all the points in each component with large cardinality as a subcluster, and then, similar to density peaks clustering, assign each remaining point to its corresponding existing subcluster. Extensive experiments on eight publicly available datasets demonstrate the effectiveness of our clustering algorithm.

A Deep Clustering Algorithm based on Gaussian Mixture Model

Clustering autonomously learns the implicit cluster structure in the original data without prior knowledge. The effect of ordinary clustering algorithms is not good to cluster high-dimensional data. In this paper, we propose a deep clustering algorithm based on Gaussian mixture model, which combines two models of stacked auto-encoder and Gaussian mixture model. This algorithm uses the expectation maximization algorithm of reducing dimension data feature to train Gaussian mixture and updates the data cluster so that the data is clustered in the feature space. The experimental results demonstrate that the proposed algorithm improves the clustering accuracy, and verifies the effectiveness of the algorithm.

ISVS3CE: Incremental Support Vector Semi-Supervised Subspace Clustering Ensemble and ENhanced Bat Algorithm (ENBA) for High Dimensional Data Clustering

In the recent work, Incremental Soft Subspace Based Semi-Supervised Ensemble Clustering (IS4EC) framework was proposed which helps in detecting clusters in the dataset. IS4EC framework also increases the results of clustering by reducing the intra-cluster distance and increasing the inter-cluster distance with increased cluster quality. It cannot attain acceptable results while handling high dimensional data. However, decreasing the dimensional subspace becomes extremely difficult issue. In IS4EC framework, to choose the optimal ensemble members also extremely becomes challenging issue. In order to solve these issues of traditional cluster ensemble methods, first propose an Incremental Support vector Semi-Supervised Subspace Clustering Ensemble (ISVS3CE) framework which makes utilized of benefits of the random subspace algorithm and the Constraint Propagation (CP) algorithm. Here the centroid values were selected by using the Support Vector Machine (SVM) classifier. In the ISVS3CE framework, Incremental Ensemble Member Chosen (IEMC) process is performed by using the ENhanced Bat Algorithm (ENBA), and the normalized cut algorithm is introduced to perform high dimensional data clustering. The ISVS3CE framework is successful for solving high dimensional data issue, at the same time as the CP algorithm is valuable for incorporating the prior information. Results demonstrate that the proposed ISVS3CE framework performs well on datasets by means of very high dimensionality, and better than the traditional clustering ensemble methods.

An intelligent clustering algorithm for high-dimensional multiview data in big data applications

There are many high-dimensional multiview data in various big data applications. It is very difficult to deal with those high-dimensional multiview data for the classic clustering algorithms, which consider all features of data with equal relevance. To tackle this challenging problem, this paper aims at proposing a novel intelligent weighting k-means clustering (IWKM) algorithm based on swarm intelligence. Firstly, the degree of coupling between clusters is presented in the model of clustering to enlarge the dissimilarity of clusters. Various weights of views and features are used in the weighting distance function to determine the clusters of objects. Secondly, to eliminate the sensitivity of initial cluster centers, swarm intelligence is utilized to find initial cluster centers, weights of views, and weights of features by a global search. Lastly, a precise perturbation is proposed to improve optimization performance of swarm intelligence. To verify the performance of clustering for high-dimensional multiview data, the experiments were performed by the evaluation metrics of Rand Index, Jaccard Coefficient and Folkes Russe in five big data applications on the two different computational platforms of apache spark and single node. The experimental results show that IWKM is effective and efficient in clustering of high-dimensional multiview data, and can obtain better performance than the other 5 kinds of approaches in these complicated data sets with more views and higher dimensions on apache spark and single node.

A Free Energy Based Approach for Distance Metric Learning.

We present a reformulation of the distance metric learning problem as a penalized optimization problem, with a penalty term corresponding to the von Neumann entropy of the distance metric. This formulation leads to a mapping to statistical mechanics such that the metric learning optimization problem becomes equivalent to free energy minimization. Correspondingly, our approach leads to an analytical solution of the optimization problem based on the Boltzmann distribution. The mapping established in this work suggests new approaches for dimensionality reduction and provides insights into determination of optimal parameters for the penalty term. Furthermore, we demonstrate that the metric projects the data onto direction of maximum dissimilarity with optimal and tunable separation between classes and thus the transformation can be used for high dimensional data visualization, classification, and clustering tasks. We benchmark our method against previous distance learning methods and provide an efficient implementation in an R package available to download at: https://github.com/kouroshz/fenn.

High Dimensional Data Clustering using Self-Organized Map

As the population grows and e economic development, houses could be one of basic needs of every family. Therefore, housing investment has promising value in the future. This research implements the Self-Organized Map (SOM) algorithm to cluster house data for providing several house groups based on the various features. K-means is used as the baseline of the proposed approach. SOM has higher silhouette coefficient (0.4367) compared to its comparison (0.236). Thus, this method outperforms k-means in terms of visualizing high-dimensional data cluster. It is also better in the cluster formation and regulating the data distribution.

Model-based clustering of censored data via mixtures of factor analyzers

Mixtures of factor analyzers (MFA) provide a promising tool for modeling and clustering high-dimensional data that contain an overwhelmingly large number of attributes measured on individuals arisen from a heterogeneous population. Due to the restriction of experimental apparatus, measurements can be limited to some lower and/or upper detection bounds and thus the data are possibly censored. In this paper, we extend the MFA to accommodate censored data, and the new model is called the MFA with censoring (MFAC). A computationally feasible alternating expectation conditional maximization (AECM) algorithm is developed to carry out maximum likelihood estimation of the MFAC model. Practical issues related to model-based clustering and recovery of censored data are also discussed. Simulation studies are conducted to examine the effect of censoring in classification, estimation and cluster validation. We also present an application of the proposed approach to two real data examples in which a certain number of left-censored observations are present.

Latent Feature Group Learning for High-Dimensional Data Clustering

In this paper, we propose a latent feature group learning (LFGL) algorithm to discover the feature grouping structures and subspace clusters for high-dimensional data. The feature grouping structures, which are learned in an analytical way, can enhance the accuracy and efficiency of high-dimensional data clustering. In LFGL algorithm, the Darwinian evolutionary process is used to explore the optimal feature grouping structures, which are coded as chromosomes in the genetic algorithm. The feature grouping weighting k-means algorithm is used as the fitness function to evaluate the chromosomes or feature grouping structures in each generation of evolution. To better handle the diverse densities of clusters in high-dimensional data, the original feature grouping weighting k-means is revised with the mass-based dissimilarity measure rather than the Euclidean distance measure and the feature weights are optimized as a nonnegative matrix factorization problem under the orthogonal constraint of feature weight matrix. The genetic operations of mutation and crossover are used to generate the new chromosomes for next generation. In comparison with the well-known clustering algorithms, LFGL algorithm produced encouraging experimental results on real world datasets, which demonstrated the better performance of LFGL when clustering high-dimensional data.

Efficient mixture model for clustering of sparse high dimensional binary data

Clustering is one of the fundamental tools for preliminary analysis of data. While most of the clustering methods are designed for continuous data, sparse high-dimensional binary representations became very popular in various domains such as text mining or cheminformatics. The application of classical clustering tools to this type of data usually proves to be very inefficient, both in terms of computational complexity as well as in terms of the utility of the results. In this paper we propose a mixture model, SparseMix, for clustering of sparse high dimensional binary data, which connects model-based with centroid-based clustering. Every group is described by a representative and a probability distribution modeling dispersion from this representative. In contrast to classical mixture models based on the EM algorithm, SparseMix: is specially designed for the processing of sparse data; can be efficiently realized by an on-line Hartigan optimization algorithm; describes every cluster by the most representative vector. We have performed extensive experimental studies on various types of data, which confirmed that SparseMix builds partitions with a higher compatibility with reference grouping than related methods. Moreover, constructed representatives often better reveal the internal structure of data.

SubspaceDB : In-database subspace clustering for analytical query processing

High dimensional data analysis within relational database management systems (RDBMS) is challenging because of inadequate support from SQL. Currently, subspace clustering of high dimensional data is implemented either outside DBMS using wrapper code or inside DBMS using SQL User Defined Functions/Aggregates(UDFs/UDAs). However, both these approaches have potential disadvantages from performance, resource usage, and security perspective for voluminous and frequently updated data. Hence, we propose an efficient querying system, named SubspaceDB, that implements subspace clustering directly within an RDBMS. SubspaceDB provides a novel set of query operators, each with an optimization objective, to facilitate interactive analysis for subspace clustering. The query operators focus on retrieving optimal answers to four key query types : (a) Medoid queries, (b) Neighbourhood queries, (c) Partial similarity queries, and (d) Prominence queries, that aid the formation of subspace clusters. Experimental studies on real and synthetic databases of size 15M tuples and 104 attributes show that our proposed approach SubspaceDB can be over 10 times faster as compared to a conventional wrapper-based or SQL UDF approach. The proposed approach is also efficient in retrieving at least 50% data with performance improvement of at least 25%.

Reference vector-based multi-objective clustering for high-dimensional data

Subspace clustering algorithms have shown their advantage in handling high-dimensional data by optimizing a linear combination of clustering criteria. However, setting the coefficients of these criteria items without prior knowledge will lead to inaccurate and poor robust clustering results. To address this problem, in this paper, we propose to optimize the multiple clustering criteria simultaneously without any predefined coefficients by a multi-objective evolutionary algorithm. Furthermore, to accelerate the convergence of the algorithm, we provide a novel local search method. In it, the multi-objective clustering problem is decomposed into many localized scalarizing sub-problems by reference vectors. Solutions are then locally searched around their associated sub-problems. Thirdly, we develop a knee-pruning fuzzy ensemble method for selecting the final solution. This method applies clustering ensemble in solutions selected from knee regions to get robust results. Experiments on UCI benchmarks and gene expression datasets show that our proposed algorithm can efficiently handle high-dimensional clustering problems without any user-defined coefficients.

Exploiting multi–core and many–core parallelism for subspace clustering

Abstract Finding clusters in high dimensional data is a challenging research problem. Subspace clustering algorithms aim to find clusters in all possible subspaces of the dataset, where a subspace is a subset of dimensions of the data. But the exponential increase in the number of subspaces with the dimensionality of data renders most of the algorithms inefficient as well as ineffective. Moreover, these algorithms have ingrained data dependency in the clustering process, which means that parallelization becomes difficult and inefficient. SUBSCALE is a recent subspace clustering algorithm which is scalable with the dimensions and contains independent processing steps which can be exploited through parallelism. In this paper, we aim to leverage the computational power of widely available multi-core processors to improve the runtime performance of the SUBSCALE algorithm. The experimental evaluation shows linear speedup. Moreover, we develop an approach using graphics processing units (GPUs) for fine-grained data parallelism to accelerate the computation further. First tests of the GPU implementation show very promising results.

Projected memory clustering

We present a new algorithm PMC (Projected Memory Clustering) for projected clustering of high dimensional data. It effectively discovers clusters described by affine subspaces parallel to the main axes of coordinate system. The number of clusters and dimensions of subspaces are selected automatically. Experiments performed on various types of data show that PMC detects clustering structures better than related projected clustering methods. Moreover, it is fast, which makes it suitable for practical use.

ChronoClust: Density-based clustering and cluster tracking in high-dimensional time-series data

In many scientific disciplines, the advent of new high-throughput technologies is giving rise to vast quantities of high-dimensional time-series data. A common requirement is to identify clusters of data-points with similar characteristics in this experimental data, and track their development over time. In this article we present ChronoClust, a novel density-based clustering algorithm for processing a time-series of discrete datasets, generating arbitrarily shaped clusters, and explicitly tracking their temporal evolution. We provide a conceptualisation of ChronoClust’s parameters, and guidelines for selecting their values. The development of ChronoClust was motivated by the need to characterise the immune response to disease. As such, we demonstrate and evaluate ChronoClust’s operation on two immune-related datasets: (1) a synthetic dataset exhibiting the temporal evolution qualities of the immune response as they would be observed through mass cytometry, a cutting edge high-throughput technology, and (2) a Flow cytometry dataset capturing the immune response in West Nile Virus (WNV)-infected mice. Our comprehensive qualitative and quantitative analyses confirm ChronoClust’s suitability for this type of problem: the temporal relationships engineered into the synthetic dataset are successfully recovered, and the cell populations and dynamics unveiled in the WNV dataset match those identified through a domain expert. ChronoClust is applicable beyond Immunology, and we provide an open source Python implementation to support its adoption more widely. We additionally make our two datasets publicly available to promote reproducible research and third-party work on temporal clustering and cluster tracking.

Adaptive multi-view subspace clustering for high-dimensional data

With the rapid development of multimedia technologies, we frequently confront with high-dimensional data and multi-view data, which usually contain redundant features and distinct types of features. How to efficiently cluster such kinds of data is still a great challenge. Traditional multi-view subspace clustering aims to determine the distribution of views by extra empirical parameters and search the optimal projection matrix by eigenvalue decomposition, which is impractical for real-world applications. In this paper, we propose a new adaptive multi-view subspace clustering method to integrate heterogenous data in the low-dimensional feature space. Concretely, we extend K-means clustering with feature learning to handle high-dimensional data. Besides, for multi-view data, we evaluate the weights of distinct views according to their compactness of the cluster structure in the low-dimensional subspace. We apply the proposed method to four benchmark datasets and compare it with several widely used clustering algorithms. Experimental results demonstrate the effectiveness of the proposed method.

Self-weighted Multi-view Subspace Clustering With Low Rank Tensor Constraint

Manufacturing process is making a difference in the Product-Service Systems (PSS). In the manufacturing process, the real-time high-dimensional data will be generated by the production line. In the field of high-dimensional data analysis, a very important analysis method is high-dimensional data clustering, and subspace clustering is one of the effective methods of high-dimensional data clustering. Clustering is an important topic in data mining. In real world, the data represented with multi view information makes the clustering algorithms more difficult. In this paper, we explore an improved multi-view clustering algorithm. Unlike the conventional multi-view subspace clustering methods that usually average the weight of views, the proposed algorithm simultaneously takes into account the expression and weight of each view. Specifically, every view is re-represented with the complementary information from multiple views by low rank tensor, and the views are fused by exploring a Laplacian rank constrained graph. With the complementary information from multiple views and the different weights of the views, the proposed algorithm is improved. Experimental results on various real-world datasets demonstrate the effectiveness of our research.

Hyperspectral Mineral Target Detection Based on Density Peak

Hyperspectral remote sensing, with its narrow band imaging, provides the potential for fine identification of ground objects, and has unique advantages in mineral detection. However, the image is nonlinear and the pure pixel is scarce, so using standard spectrum detection will lead to an increase of the number of false alarm and missed detection. The density peak algorithm performs well in high-dimensional space and data clustering with irregular category shape. This paper used the density peak clustering to determine the cluster centers of various categories of images, and took it as the target spectrum, and took the clustering results as the ground data. Two methods of HUD and OSP were used to detect the image, and the correlation coefficients of the spectrum of each cluster center and the mineral spectrum of the spectral library were obtained. Finally, the results were compared with the mapping results of Clark et al. The experimental results showed that the cluster center spectrum as the target can well detected the distribution of the corresponding minerals, and it has higher correlation coefficient with mineral in the result of mapping.

Dynamic Feature Selection for Clustering High Dimensional Data Streams

Change in a data stream can occur at the concept level and at the feature level. Change at the feature level can occur if new, additional features appear in the stream or if the importance and relevance of a feature changes as the stream progresses. This type of change has not received as much attention as concept-level change. Furthermore, a lot of the methods proposed for clustering streams (density-based, graph-based, and grid-based) rely on some form of distance as a similarity metric and this is problematic in high-dimensional data where the curse of dimensionality renders distance measurements and any concept of “density” difficult. To address these two challenges we propose combining them and framing the problem as a feature selection problem, specifically a dynamic feature selection problem. We propose a dynamic feature mask for clustering high dimensional data streams. Redundant features are masked and clustering is performed along unmasked, relevant features. If a feature's perceived importance changes, the mask is updated accordingly; previously unimportant features are unmasked and features which lose relevance become masked. The proposed method is algorithm-independent and can be used with any of the existing density-based clustering algorithms which typically do not have a mechanism for dealing with feature drift and struggle with high-dimensional data. We evaluate the proposed method on four density-based clustering algorithms across four high-dimensional streams; two text streams and two image streams. In each case, the proposed dynamic feature mask improves clustering performance and reduces the processing time required by the underlying algorithm. Furthermore, change at the feature level can be observed and tracked.

Model selection and application to high-dimensional count data clustering

EDCM, the Exponential-family approximation to the Dirichlet Compound Multinomial (DCM), proposed by Elkan (2006), is an efficient statistical model for high-dimensional and sparse count data. EDCM models take into account the burstiness phenomenon correctly while being many times faster than DCM. This work proposes the use of Minimum Message Length (MML) criterion for determining the number of components that best describes the data with a finite EDCM mixture model. Parameters estimation is based on the previously proposed Deterministic Annealing Expectation-Maximization (DAEM) approach. The validation of the proposed unsupervised algorithm involves different real applications: text document modeling, topic novelty detection and hierarchical image clustering. A comparison with results obtained for other information-theory based selection criteria is provided.

Improved image clustering with deep semantic embedding

Dimensionality reduction has found extensive use in the area of high dimensional data clustering. But unfortunately some valuable discriminative information might be lost in the process of dimensionality reduction, including feature extraction and feature selection. This issue inevitably degrades the performance of clustering algorithms. Recently, the semantic space embedding is emerging as a promising technique for state-of-the-art clustering methods, which can provide extra discriminative information and reasonably improve the clustering performance. In this paper, we plan to improve the performance of high dimensional image clustering by embedding semantic information into the original visual space. Inspired by the great success of deep learning, we employed a multi-layer autoencoder based on deep neural networks (DNNs) to undertake the semantic feature embedding and dimensionality reduction. By this way, the final image clustering task is carried out in the lower-dimensional feature space with deep semantic embedding. A series of experiments on acknowledged benchmark image datasets demonstrate that the proposed approaches can achieve superior performance over several existing clustering methods.