Related Clustering Methods Research Articles

Limitations of clustering with PCA and correlated noise

It is now common to have a modest to large number of features on individuals with complex diseases. Unsupervised analyses, such as clustering with and without preprocessing by Principle Component Analysis (PCA), is widely used in practice to uncover subgroups in a sample. However, in many modern studies features are often highly correlated and noisy (e.g. SNP's, -omics, quantitative imaging markers, and electronic health record data). The practical performance of clustering approaches in these settings remains unclear. Through extensive simulations and empirical examples applying Gaussian Mixture Models and related clustering methods, we show these approaches (including variants of kmeans, VarSelLCM, HDClassifier, and Fisher-EM) can have very poor performance in many settings. We also show the poor performance is often driven by either an explicit or implicit assumption by the clustering algorithm that high variance features are relevant while lower variance features are irrelevant, called the variance as relevance assumption. We develop practical pre-processing approaches that improve analysis performance in some cases. This work offers practical guidance on the strengths and limitations of unsupervised clustering approaches in modern data analysis applications.

Coordinate Descent Optimized Trace Difference Model for Joint Clustering and Feature Extraction

Joint clustering and dimensionality reduction methods are a promising solution to clustering due to its scalability to high-dimensional data. Some methods leverage trace ratio criterion and attain clusters by borrowing the K-means algorithm. However, trace ratio criterion has no close-formed solution for the discriminative projection matrix and the K-means algorithm has a limited capacity to handle the many-cluster problem. In this paper, Coordinate Descent Optimized Trace Difference model (CDOTD) is proposed for joint clustering and feature extraction. Formulating the objective function as a direct trace difference criterion containing a balance parameter, CDOTD harmonizes between-cluster scatter maximization and within-cluster scatter minimization by the balance parameter. Using the direct trace difference criterion, CDOTD can straightforward solve for the discriminative projection matrix and avoid obtaining a poor discriminative projection matrix in the iterative manner when a bad cluster start is given. CDOTD uses the coordinate descent method for clustering optimization, improving the ability to address the many-cluster problem. Extensive experiments show that CDOTD has achieved significant performance improvements compared to previous trace ratio criterion related joint clustering and feature extraction methods, and also outperformed other clustering methods in most cases.

Local-Sample-Weighted Clustering Ensemble with High-Order Graph Diffusion

The clustering ensemble method has attracted much attention because it can improve the stability and robustness of single clustering methods. Among them, similarity-matrix-based methods or graph-based methods have had a wide range of applications in recent years. Most similarity-matrix-based methods calculate fully connected pairwise similarities by treating a base cluster as a whole and ignoring the importance of the relevance ranking of samples within the same base cluster. Since unreliable similarity estimates degrade clustering performance, constructing accurate similarity matrices is of great importance in applications. Higher-order graph diffusion based on reliable similarity matrices can further uncover potential connections between data. In this paper, we propose a more substantial graph-learning-based ensemble algorithm for local-sample-weighted clustering, which implicitly optimizes the adaptive weights of different neighborhoods based on the ranking importance of different neighbors. By further diffusion on the consensus matrix, we obtained an optimal consistency matrix with more substantial discriminative power, revealing the potential similarity relationship between samples. The experimental results showed that, compared with the second-best DREC algorithm, the accuracy of the proposed algorithm improved by 17.7%, and that of the normalized mutual information (NMI) algorithm improved by 15.88%. All empirical results showed that our clustering model consistently outperformed the related clustering methods.

A new robust fuzzy clustering framework considering different data weights in different clusters

In conventional fuzzy C-means clustering algorithms, each data and each feature are treated equally, the clustering performance is sensitive to the noise points; in existing weighting clustering algorithms, few studies have focus on data weighting and feature weighting simultaneously, besides, the same data in different clusters is treated equally. To address this issue, in this paper, taking the different data weights in different clusters and the different feature weights in different clusters into consideration, we present a new robust fuzzy C-means clustering framework. For the first time, we propose a whole new idea that the same data in different clusters should have different importance, the different data in a cluster should have different importance, as well; By the new data weighting method, the proposed clustering algorithm can weaken the impact of noise points on the formation of each clustering center, which could enhance the robustness of clustering; to stimulate more data and more features to take part in the process of clustering and to avoid overfitting, we add l2-norm regularization of the data weights and l2-norm regularization of the feature weights to the objective function. Then, based on the presented objective function, we get the scientific update rules of the different data weights in different clusters, the different feature weights in different clusters, the membership degrees, and the cluster centers, during each iteration. To assess the performance of the new fuzzy C-means framework, experimental verifications on synthetic dataset and real-world datasets are conducted, experimental results have shown that the new algorithm can achieve better clustering performances in comparison to other related clustering methods.

Applying the Hierarchical Gray Relational Clustering Method to Municipal Water Use in Turkey

This study aims to compare the municipal water distribution performance and classify the municipal water distribution systems in the provinces of Turkey using the Gray Relational Hierarchical clustering method for both 2006 and 2016. A correlation analysis was used to determine the variables affecting the distributed water. The Hierarchical gray clusters and the changes during the decade are presented with spatial distribution maps using statistically significant variables. Consequently, the findings reveal that the municipalities of Istanbul, Izmir, Bursa, Gaziantep, and Kocaeli managed water distribution effectively, whereas the municipalities of Konya, Şanlıurfa, Diyarbakır, Samsun, Trabzon, and Sakarya managed it poorly.

Nearest Descent, In-Tree, and Clustering

Clustering aims at discovering the natural groupings in a dataset, prevalent in many disciplines that involve multivariate data analysis. In this paper, we propose a physically inspired graph-theoretical clustering method, which first makes the data points organized into an attractive graph, called In-Tree, via a physically inspired rule, called Nearest Descent (ND). The rule of ND works to select the nearest node in the descending direction of potential as the parent node of each node, which is fundamentally different from the classical Gradient Descent. The constructed In-Tree proves a very good candidate for clustering due to its particular features and properties. In the In-Tree, the original clustering problem is reduced to a problem of removing the inter-cluster edges from this graph. Pleasingly, those inter-cluster edges are usually so distinguishable that they can be easily determined by different automatic edge-cutting methods. We also propose a visualized strategy to validate the effectiveness of the automatic edge-cutting methods. The experimental results reveal that the proposed method is superior to the related clustering methods. The results also reveal the characteristics of different automatic cutting methods and the meaningfulness of the visualized strategy in increasing the reliability of the clustering results in practice.

Transfer Collaborative Fuzzy Clustering in Distributed Peer-to-Peer Networks

The traditional collaborative fuzzy clustering can effectively perform data clustering in distributed peer-to-peer networks, which is an impossible task to complete for the centralized clustering methods due to privacy and security requirements or network transmission technology constraints. But it will increase the number of clustering iterations and lead to lower efficiency of the clustering. Moreover, the collaborative mechanism hidden in the iterative process of clustering cannot be well revealed and explained. In this article, a novel series of transfer collaborative fuzzy clustering algorithms are proposed to solve these issues. In the first basic algorithm, the transfer learning among neighbor nodes vividly expresses the collaborative mechanism and enhances the information collaboration to accelerate the convergence of fuzzy clustering. Meanwhile, neighbor nodes can learn the knowledge from each other to further promote their respective clustering performance. Then, an improved version, with the learning-rate-adjustable strategy instead of fixed values, is designed to highlight the different influence between neighbor nodes, and the appropriate learning rates between neighbor nodes are achieved to ensure the stable clustering accuracy. Finally, two extended versions with the attribute-weight-entropy regularization technique are presented for the clustering of high dimensional sparse data and the extraction of important subspace features. Experiments show the efficiency of the proposed algorithms compared with the related prototype-based clustering methods.

Research on Grey Relational Clustering Model of Multiobjective Human Resources Based on Time Constraint

The existing grey relational clustering method has limitations in the application of multidimensional sequences and cannot directly calculate the grey correlation degree between unequal-length sequences. In this paper, by introducing the multidimensional dynamic DTW distance into the existing 3D grey relational model, a new grey relational analysis model that can be applied to multidimensional data is proposed, which is based on DTW distance. The model does not require one-to-one correspondence of data points but evaluates the similarity of its geometric curves by calculating the shortest distance between sequences. In addition, since the traditional grey correlation clustering method is implemented, the method first extracts the reference sequence from the observation sequence and then calculates the similarity between the observation objects by calculating the grey correlation degree between each sequence and the reference sequence, so each object only needs to be calculated once. The experimental results show that the multidimensional grey correlation degree based on DTW distance and the grey relational clustering model oriented to multidimensional data are more accurate than other existing methods. Finally, the grey relation clustering method of multidimensional data is used to analyze the multiobjective human resource grey relational clustering model under time constraints, and the validity of the model is verified.

Robust multi-view k-means clustering with outlier removal

Contemporary datasets are often comprised of multiple views of data, which provide complete and complementary information in different views, and multi-view clustering is one of the most crucial techniques in multi-view data analysis. However, traditional multi-view clustering methods are sensitive to noises and outliers, suffering from severe performance degradation when the dataset contains many outliers. Moreover, the commonly used multi-view clustering methods are restricted by high time complexity. To address these problems, we propose a robust multi-view k-means algorithm with outlier detection, i.e., Multi-View Clustering with Outlier Removal (MVCOR). This method is designed to remove the outliers and thus boosts the clustering performance on multi-view data with low time complexity. By defining two types of outliers, MVCOR uses the well-defined outlier removal strategy to categorize all the outliers into two specific clusters and performs robust clustering on the clean data at the same time. This strategy significantly improves the clustering performance as well as the model robustness, making MVCOR a more practical approach for real-world scenarios. Besides, the proposed model is efficiently optimized by a well-designed alternating minimization algorithm which is strictly proved to be convergent. Extensive experiments on both synthetic and real-world datasets demonstrate that MVCOR consistently outperforms the related clustering methods on clustering performance as well as robustness to outliers, and achieves comparable performance to the state-of-the-art multi-view outlier detection methods.

Learning Adaptive Neighborhood Graph on Grassmann Manifolds for Video/Image-Set Subspace Clustering

The objective of self-expression based spectral clustering is to learn an affinity matrix which accurately reflects the similarity among data, and the Laplacian constraint is usually exploited to make the affinity matrix preserve the global structure of raw data. However, there exist two drawbacks: firstly, these methods are mostly designed for vectorial data in Euclidean spaces, which are not suitable for multidimensional data with nonlinear manifold structure, e.g., videos and image-sets. Secondly, the clustering performance heavily relies on the quality of a pre-learned Laplacian matrix in which the global structure may be mis-interpreted without considering manifold structures. In this paper, we firstly provide a unified framework about self-expression learning on Grassmann manifolds, which implements the clustering tasks for multidimensional data under subspace views. Then, to assign optimal neighbors to each data depending on the local distance, we adaptively learn the neighborhood relationship from the obtained self-expression coefficient matrix, referred to Learning Adaptive Neighborhood Graph on Grassmann manifolds (GMAN). In the optimization process, the neighborhood relationship can be adaptively learned and updated with the coefficient matrix. The experimental results on five public datasets show that the proposed method is obviously better than many related clustering methods based on Grassmann manifolds, proving the effectiveness of GMAN in multidimensional data clustering.

Cautious relational clustering: A thresholding approach

We propose in this article a new relational clustering method that can return a partial answer (i.e., a set of clusterings) in some cases. Starting from relational or similarity data, we determine a partial equivalence relation defined on the set of objects (two objects are linked if they belong to the same cluster): the key idea is to allow the method to abstain on some pairwise links because they cannot be determined with enough certainty from the data. This cautious equivalence relation represents a set of possible hard clusterings which can be obtained by completing the partial relation. This formalization makes it possible to easily detect ambiguous links and to identify subsets of objects with uncertain relationship. We illustrate the potential interest of our approach as a tool for exploratory data analysis of synthetic and real data sets.

Unsupervised fuzzy model-based Gaussian clustering

In 1993, Banfield and Raftery first proposed model-based Gaussian (MB-Gauss) clustering, using eigenvalue decomposition of Gaussian covariance matrix to detect different cluster shapes. In this paper, we extend MB-Gauss to a fuzzy model-based Gaussian (F-MB-Gauss) clustering. However, the performance of both MB-Gauss and F-MB-Gauss algorithms depend heavily on initializations and require a number of clusters to be assigned a priori. To solve these problems, an unsupervised learning schema for F-MB-Gauss clustering is proposed. Therefore, an unsupervised fuzzy model-based Gaussian (UF-MB-Gauss) clustering algorithm is constructed where the proposed UF-MB-Gauss algorithm not only solves the initialization problem, but can also automatically obtain an optimal number of clusters. In the literature, MB-Gauss with Bayesian information criterion (BIC) is a common way for determining the number of clusters for Gaussian mixtures. Furthermore, several clustering methods that can automatically determine the number of clusters have been proposed in the literature. Therefore, comparison is made between the proposed UF-MB-Gauss clustering algorithm and these related clustering methods using numerical data and real data sets. Experimental results and comparisons demonstrate the superiority and usefulness of the proposed UF-MB-Gauss clustering algorithm.

New automatic fuzzy relational clustering algorithms using multi-objective NSGA-II

Most of the fuzzy clustering methods use only centroid information and cannot differentiate the geometric structures of clusters due to the cohesion and separation measures of the fuzzy partition. Moreover, conventional clustering methods confine the search space of automatic fuzzy clustering as it shows a tendency to fall into local minima as well as the number of clusters is required to be provided as prior knowledge. In this paper, combining fuzzy relational clustering (FRC) with multi-objective genetic algorithm NSGA-II, we propose two new fuzzy relational clustering methods, referred to as FRCNSGA and IFRCNSGA. In IFRCNSGA, NSGA-II is used to adjust the parameters of FRC algorithm dynamically such as the appropriate number of clusters and the initial membership values. Thus this proposed model handles the intricacy of automatic clustering a dataset, at the same time, avoids the problem of falling into local minima quickly. For both of the proposed models, NSGA-II is used to optimize two cluster validity indices named, separation, and cohesion, simultaneously, whereas, FRC is used to handle the overlapping properties of clusters. In this case, fuzzy membership degrees are used to compute the overlap-separation. To eliminate the impediment of automatic fuzzy clustering, it is necessary to optimize one or more criteria. Therefore, we consider NSGA-II as an optimization algorithm to optimize more than single criterion. The experimental results have exhibited that the intended methods provide competitive results in the compound, overlapped, high-dimensional gene expression and non-gene expression datasets. The proposed methods are able to determine well-separated, hyperspherical, noncompact and overlapping clusters compared to other existing methods.

A Granular Self-Organizing Map for Clustering and Gene Selection in Microarray Data.

A new granular self-organizing map (GSOM) is developed by integrating the concept of a fuzzy rough set with the SOM. While training the GSOM, the weights of a winning neuron and the neighborhood neurons are updated through a modified learning procedure. The neighborhood is newly defined using the fuzzy rough sets. The clusters (granules) evolved by the GSOM are presented to a decision table as its decision classes. Based on the decision table, a method of gene selection is developed. The effectiveness of the GSOM is shown in both clustering samples and developing an unsupervised fuzzy rough feature selection (UFRFS) method for gene selection in microarray data. While the superior results of the GSOM, as compared with the related clustering methods, are provided in terms of β -index, DB-index, Dunn-index, and fuzzy rough entropy, the genes selected by the UFRFS are not only better in terms of classification accuracy and a feature evaluation index, but also statistically more significant than the related unsupervised methods. The C-codes of the GSOM and UFRFS are available online at http://avatharamg.webs.com/software-code.

Visualizing Web Search Results on the Basisof Snippet Similarity

Search engines are programs that search documents for specified keywords and return list of documents where the keywords were found. Textual snippet based search enable us to search documents in a better way. Search tasks would be easier if users were given an overview of document clusters in an organized manner so as to show how related they are content wise. In this paper, we introduce a novel multi-viewpoint based similarity measure and two related clustering methods. Using multiple viewpoints, more informative assessment of similarity could be achieved. We also propose a visualization technique to display the results of web queries.

Subspace K-means clustering

To achieve an insightful clustering of multivariate data, we propose subspace K-means. Its central idea is to model the centroids and cluster residuals in reduced spaces, which allows for dealing with a wide range of cluster types and yields rich interpretations of the clusters. We review the existing related clustering methods, including deterministic, stochastic, and unsupervised learning approaches. To evaluate subspace K-means, we performed a comparative simulation study, in which we manipulated the overlap of subspaces, the between-cluster variance, and the error variance. The study shows that the subspace K-means algorithm is sensitive to local minima but that the problem can be reasonably dealt with by using partitions of various cluster procedures as a starting point for the algorithm. Subspace K-means performs very well in recovering the true clustering across all conditions considered and appears to be superior to its competitor methods: K-means, reduced K-means, factorial K-means, mixtures of factor analyzers (MFA), and MCLUST. The best competitor method, MFA, showed a performance similar to that of subspace K-means in easy conditions but deteriorated in more difficult ones. Using data from a study on parental behavior, we show that subspace K-means analysis provides a rich insight into the cluster characteristics, in terms of both the relative positions of the clusters (via the centroids) and the shape of the clusters (via the within-cluster residuals).

Clustering with Multiviewpoint-Based Similarity Measure

All clustering methods have to assume some cluster relationship among the data objects that they are applied on. Similarity between a pair of objects can be defined either explicitly or implicitly. In this paper, we introduce a novel multiviewpoint-based similarity measure and two related clustering methods. The major difference between a traditional dissimilarity/similarity measure and ours is that the former uses only a single viewpoint, which is the origin, while the latter utilizes many different viewpoints, which are objects assumed to not be in the same cluster with the two objects being measured. Using multiple viewpoints, more informative assessment of similarity could be achieved. Theoretical analysis and empirical study are conducted to support this claim. Two criterion functions for document clustering are proposed based on this new measure. We compare them with several well-known clustering algorithms that use other popular similarity measures on various document collections to verify the advantages of our proposal.

Applying hierarchical grey relation clustering analysis to geographical information systems – A case study of the hospitals in Taipei City

Deng proposed grey clustering analysis (GCA) in 1987. Later, Jin presented a new method in 1993, called grey relational clustering (GRC) method that combined grey relational analysis with clustering. However, the GRC method cannot use a tree diagram to make appropriate classification decisions without re-computation. This study thus attempts to combine GRC and hierarchical clustering analysis. Given the existence of an excess of medical resources in the Taipei area, this study attempts to understand the degree of concentration of medical resources in this area. Specifically, this study applies a geographical information system (GIS) to present the geographical distribution of hospitals in Taipei. Additionally, a new-type of cluster analysis, known as hierarchical grey relation clustering analysis, is used to analyze the distribution of hospitals and understand how they compete with one another. The analytical results demonstrate that hierarchical grey relation clustering analysis is a suitable method of analyzing geographical position. Tree diagrams can help policymakers make appropriate classification decisions without re-computation. The study results can inform hospitals of their competitors and help them to develop appropriate responses. Additionally, the analytical results can also provide a reference to government or hospital policymakers to help them position hospitals in areas, thus achieving a better distribution of medical resources in Taipei.

Research for RBF Neural Networks Modeling Accuracy of Determining the Basis Function Center Based on Clustering Methods

The radial basis function (RBF) neural network is superior to other neural network on the aspects of approximation ability, classification ability, learning speed and global optimization etc., it has been widely applied as feedforward networks, its performance critically rely on the choice of RBF centers of network hidden layer node. K-means clustering, as a commonly method used on determining RBF center, has low neural network generalization ability, due to its clustering results are not sensitive to initial conditions and ignoring the influence of dependent variable. In view of this problem, fuzzy clustering and grey relational clustering methods are proposed to substitute K-means clustering, RBF center is determined by the results of fuzzy clustering or grey relational clustering, and some researches of RBF neural networks modeling accuracy are done. Practical modeling cases demonstrate that the modeling accuracy of fuzzy clustering RBF neural networks and grey relational clustering RBF neural networks are significantly better than K-means clustering RBF neural networks, applying of fuzzy clustering or grey relational clustering to determine the basis function center of RBF neural networks hidden layer node is feasible and effective.

Analysis of infrared images based on grey system and neural network

PurposeThe purpose of this paper is to develop a system to analyse the characteristics of infrared objects.Design/methodology/approachAccording to the gray scale of image pixel by the image entropy, gray scale estimating is carries on to construct the neural networks. And then the grey relational analysis and grey clustering methods are applied to filter the possible object. The target is predicted through image segmentation pretreatment based on the forecasting value by grey system and assigned corresponding mark. The forecasting precision is greatly elevated by GM (1, 1) model.FindingsThe paper illustrates that, based on the analysis and its experimental results, this system has a good recognition rate for infrared target.Research limitations/implicationsThis paper provides a way to grasp the minutial feature of the image. The filtering operation based on pixel level provided auto‐adapted filtering with a new stage.Practical implicationsApplications of grey theory deepened the content of detecting infrared targets and enriched technology of image processing.Originality/valueThis system introduces an effective method for detecting infrared targets.