Existing Clustering Approaches Research Articles

Overlapping clustering of gene expression data using penalized weighted normalized cut.

Clustering has been widely conducted in the analysis of gene expression data. For complex diseases, it has played an important role in identifying unknown functions of genes, serving as the basis of other analysis, and others. A common limitation of most existing clustering approaches is to assume that genes are separated into disjoint clusters. As genes often have multiple functions and thus can belong to more than one functional cluster, the disjoint clustering results can be unsatisfactory. In addition, due to the small sample sizes of genetic profiling studies and other factors, there may not be sufficient evidence to confirm the specific functions of some genes and cluster them definitively into disjoint clusters. In this study, we develop an effective overlapping clustering approach, which takes account into the multiplicity of gene functions and lack of certainty in practical analysis. A penalized weighted normalized cut (PWNCut) criterion is proposed based on the NCut technique and an norm constraint. It outperforms multiple competitors in simulation. The analysis of the cancer genome atlas (TCGA) data on breast cancer and cervical cancer leads to biologically sensible findings which differ from those using the alternatives. To facilitate implementation, we develop the function pwncut in the R package NCutYX.

A bounded-size clustering algorithm on fully-dynamic streaming graphs

Many contemporary data sources in a variety of domains can naturally be represented as fully-dynamic streaming graphs. How to design an efficient online streaming clustering algorithm on such graphs is of great concern. However, existing clustering approaches are inappropriate for this specific task because: (1) static clustering approaches require expensive computational cost to cluster the graph for each update and (2) the existing streaming clustering neither could fully support insertion/deletion of edges nor take temporal information into account. To tackle these issues, in this work, firstly we propose an appropriate streaming clustering model and design two new core components: streaming reservoir and cluster manager. Then we present an evolution-aware bounded-size clustering algorithm to handle the edge additions/deletions. It requires the clusters to satisfy the maximum cluster-size constraint, and maintains the recency of edges in the temporal sequence and gives high priority to the recent edges in each cluster. The experimental results show that the proposed BSC algorithm outperforms current online algorithms and is capable to keep track of the evolution of graphs. Furthermore, it obtains almost one order of magnitude higher throughput than the state-of-the-art algorithms.

EEC-FM: Energy Efficient Clustering based on Firefly and Midpoint Algorithms in Wireless Sensor Network

Wireless sensor networks (WSNs) consist of set of sensor nodes. These sensor nodes are deployed in unattended area which are able to sense, process and transmit data to the base station (BS). One of the primary issues of WSN is energy efficiency. In many existing clustering approaches, initial centroids of cluster heads (CHs) are chosen randomly and they form unbalanced clusters, results more energy consumption. In this paper, an energy efficient clustering protocol to prevent unbalanced clusters based on firefly and midpoint algorithms called EEC-FM has been proposed, where midpoint algorithm is used for initial centroid of CHs selection and firefly is used for cluster formation. Using residual energy and Euclidean distance as the parameters for appropriate cluster formation of the proposed approach produces balanced clusters to eventually balance the load of CHs and improve the network lifetime. Simulation result shows that the proposed method outperforms LEACH-B, BPK-means, Park’s approach, Mk-means, and EECPK-means with respect to balancing of clusters, energy efficiency and network lifetime parameters. Simulation result also demonstrate that the proposed approach, EEC-FM protocol is 45% better than LEACH-B, 17.8% better than BPK-means protocol, 12.5% better than Park’s approach, 9.1% better than Mk-means, and 5.8% better than EECPK-means protocol with respect to the parameter half energy consumption (HEC).

Unified Spectral Clustering With Optimal Graph

Spectral clustering has found extensive use in many areas. Most traditional spectral clustering algorithms work in three separate steps: similarity graph construction; continuous labels learning; discretizing the learned labels by k-means clustering. Such common practice has two potential flaws, which may lead to severe information loss and performance degradation. First, predefined similarity graph might not be optimal for subsequent clustering. It is well-accepted that similarity graph highly affects the clustering results. To this end, we propose to automatically learn similarity information from data and simultaneously consider the constraint that the similarity matrix has exact c connected components if there are c clusters. Second, the discrete solution may deviate from the spectral solution since k-means method is well-known as sensitive to the initialization of cluster centers. In this work, we transform the candidate solution into a new one that better approximates the discrete one. Finally, those three subtasks are integrated into a unified framework, with each subtask iteratively boosted by using the results of the others towards an overall optimal solution. It is known that the performance of a kernel method is largely determined by the choice of kernels. To tackle this practical problem of how to select the most suitable kernel for a particular data set, we further extend our model to incorporate multiple kernel learning ability. Extensive experiments demonstrate the superiority of our proposed method as compared to existing clustering approaches.

Merge or Not? Learning to Group Faces via Imitation Learning

Face grouping remains a challenging problem despite the remarkable capability of deep learning approaches in learning face representation. In particular, grouping results can still be egregious given profile faces and a large number of uninteresting faces and noisy detections. Often, a user needs to correct the erroneous grouping manually. In this study, we formulate a novel face grouping framework that learns clustering strategy from ground-truth simulated behavior. This is achieved through imitation learning (a.k.a apprenticeship learning or learning by watching) via inverse reinforcement learning (IRL). In contrast to existing clustering approaches that group instances by similarity, our framework makes sequential decision to dynamically decide when to merge two face instances/groups driven by short- and long-term rewards. Extensive experiments on three benchmark datasets show that our framework outperforms unsupervised and supervised baselines.

DHeat: A Density Heat-Based Algorithm for Clustering With Effective Radius

Density-based clustering is one of the most popular paradigms of existing clustering approaches, most approaches of this kind, such as DBSCAN, recognize clusters of data characterized by a fixed scanning radius. However, some flaws are caused by the fixed scanning radius, e.g., the determination of a proper scanning radius is nontrivial. In order to solve these problems, we revise DBSCAN, Meanshift, DPeak, etc. based on two new features, i.e., effective radius and density heat (DHeat). Generally, we name these revised clustering algorithms as DHeat. The underlying idea is based on two assumptions: 1) the existence of clusters is raised by the nonuniformity of data distribution, and the density of one data point within its r-neighborhood is proportional to the volume of the neighborhood provided the density distribution is uniform and 2) each cluster can be divided into different density layers, such as edges, shallow inner, deep inner, etc.; the deeper inner of a point locates, the higher density of that point. The experiments conducted on various test cases show that the advantage of DHeat lies in its good performance and the self-adapting scanning radius.

Alternating Relaxed Twin Bounded Support Vector Clustering

Maximum margin clustering (MMC) and its improved version are based on the spirit of support vector machine, which will inevitably lead to prohibitively computational complexity when these learning models are encountered with an enormous amount of patterns. To accelerate the clustering efficiency, we propose alternating twin bounded support vector clustering to decompose the original large problem in MMC and its variants into two smaller sized ones, in which solving expensive semi-definite programming is avoided by performing alternating optimization between cluster-specific model parameters and instance-specific labeling assignments. Also the structural risk minimization principle is implemented to obtain good generalization. Additionally, in order to avoid premature convergence, a relaxed version of our algorithm is proposed, in which the hinge loss in the original twin bounded support vector machine is replaced with the Laplacian loss. These two versions can be easily extended to the nonlinear context via kernel tricks. To investigate the efficacy of our clustering algorithm, several experiments are conducted on a number of synthetic and real-world datasets. Experimental results demonstrate that the proposed method has better performance than other existing clustering approaches in terms of clustering accuracy and time efficiency and also possesses the powerful ability to process larger-scaled datasets.

Discrete Nonnegative Spectral Clustering

Spectral clustering has been playing a vital role in various research areas. Most traditional spectral clustering algorithms comprise two independent stages (e.g., first learning continuous labels and then rounding the learned labels into discrete ones), which may cause unpredictable deviation of resultant cluster labels from genuine ones, thereby leading to severe information loss and performance degradation. In this work, we study how to achieve discrete clustering as well as reliably generalize to unseen data. We propose a novel spectral clustering scheme which deeply explores cluster label properties, including discreteness, nonnegativity, and discrimination, as well as learns robust out-of-sample prediction functions. Specifically, we explicitly enforce a discrete transformation on the intermediate continuous labels, which leads to a tractable optimization problem with a discrete solution. Besides, we preserve the natural nonnegative characteristic of the clustering labels to enhance the interpretability of the results. Moreover, to further compensate the unreliability of the learned clustering labels, we integrate an adaptive robust module with $\ell _{2,p}$ loss to learn prediction function for grouping unseen data. We also show that the out-of-sample component can inject discriminative knowledge into the learning of cluster labels under certain conditions. Extensive experiments conducted on various data sets have demonstrated the superiority of our proposal as compared to several existing clustering approaches.

Sparse Regularization in Fuzzy c-Means for High-Dimensional Data Clustering.

In high-dimensional data clustering practices, the cluster structure is commonly assumed to be confined to a limited number of relevant features, rather than the entire feature set. However, for high-dimensional data, identifying the relevant features and discovering the cluster structure are still challenging problems. To solve these problems, this paper proposes a novel fuzzy c-means (FCM) model with sparse regularization (ℓq(0<q≤1)-norm regularization), by reformulating the FCM objective function into the weighted between-cluster sum of square form and imposing the sparse regularization on the weights. An algorithm is also developed to explicitly solve the proposed model. Compared with the existing clustering models, the proposed model can shrink the weights of irrelevant features (noisy features) to exact zero, and also can be efficiently solved in analytic forms when q = 1,1/2. Experiments on both synthetic and real-world data sets show that the proposed approach outperforms the existing clustering approaches.

Nonsmooth Penalized Clustering via $\ell _{p}$ Regularized Sparse Regression.

Clustering has been widely used in data analysis. A majority of existing clustering approaches assume that the number of clusters is given in advance. Recently, a novel clustering framework is proposed which can automatically learn the number of clusters from training data. Based on these works, we propose a nonsmooth penalized clustering model via lp ( ) regularized sparse regression. In particular, this model is formulated as a nonsmooth nonconvex optimization, which is based on over-parameterization and utilizes an lp -norm-based regularization to control the tradeoff between the model fit and the number of clusters. We theoretically prove that the new model can guarantee the sparseness of cluster centers. To increase its practicality for practical use, we adhere to an easy-to-compute criterion and follow a strategy to narrow down the search interval of cross validation. To address the nonsmoothness and nonconvexness of the cost function, we propose a simple smoothing trust region algorithm and present its convergent and computational complexity analysis. Numerical studies on both simulated and practical data sets provide support to our theoretical results and demonstrate the advantages of our new method.

A Clustering Approach Using Fractional Calculus-Bacterial Foraging Optimization Algorithm for k-Anonymization in Privacy Preserving Data Mining

A tremendous amount of personal data of an individual is being collected and analyzed using data mining techniques. Such collected data, however, may also contain sensitive data about an individual. Thus, when analyzing such data, individual privacy can be breached. Therefore, to preserve individual privacy, one can find numerous approaches proposed for the same in the literature. One of the solutions proposed in the literature is k-anonymity which is used along with the clustering approach. During the investigation, the authors observed that the k-anonymization based clustering approaches all the times result in the loss of information. This paper presents a fractional calculus-based bacterial foraging optimization algorithm (FC-BFO) to generate an optimal cluster. In addition to this, the authors utilize the concept of fractional calculus (FC) in the chemotaxis step of a bacterial foraging optimization (BFO) algorithm. The main objective is to improve the optimization ability of the BFO algorithm. The authors also evaluate their proposed FC-BFO algorithm, empirically, focusing on information loss and execution time as a vital metric. The experimental evaluations show that our proposed FC-BFO algorithm generates an optimal cluster with lesser information loss as compared with the existing clustering approaches.

Transferring auxiliary knowledge to enhance heterogeneous web service clustering

The growing number of web services puts forward higher requirements for searching desired web services, and clustering web services can greatly enhance the discovery of web service. Most existing clustering approaches are designed to handle long text documents. However, the descriptions of most services are in the form of short text, which impairs the quality of clustering owing to the lack of statistical information. To solve this problem, we propose a new service clustering approach based on transfer learning from auxiliary long text data obtained from Wikipedia. To handle the inconsistencies in semantics between service descriptions and auxiliary data, we introduce a novel topic model - dual tag aided latent Dirichlet allocation (DT-LDA), which jointly learns two sets of topics on the two datasets. Experimental results show the proposed approach achieves better performance than several existing approaches.

基于人工鱼群混合聚类的大型水泵分类研究

人工鱼群是一种随机搜索优化算法，具有较快的收敛速度，对问题的机理模型与描述无严格要求，具有广泛的应用范围。本文在该算法的基础上，结合传统的K-means聚类方法，提出了一种新的人工鱼群混合聚类算法(AFSMC)，并给出相对熵混合聚类优化方法。文中将几种算法应用于大型水泵数据的分类，算例分析表明，基于相对熵的AFSMC算法在准确率与时间效益上都具有较明显的优越性。 Artificial Fish Swarm (AFS) is a stochastic search optimization algorithm with a fast convergence speed, and it is widely used in different areas due to its tolerance for descriptions and mechanism model of questions. In this paper, we propose a new clustering algorithm called Artificial Fish Swarm Mixed Clustering (AFSMC) by performing a combination of AFS and similarity based K-means. And we further propose a relative entropy based optimization method for AFSMC. Finally, we evaluate our proposal by conducting classification on the real data of large scale water pump, which is a new thought for analyzing the potential correlations among the different pumps. The experimental results show that our proposal outperforms the existing clustering approaches.

K-PRSCAN: A clustering method based on PageRank

Many existing clustering approaches are difficult to cluster non-convex or non-isotropic shapes whose centroids are not highly distinguishable. In addition, most of these approaches are often sensitive to outliers and background noise. To this end, we propose a novel clustering approach called K-PRSCAN, where PageRank algorithm is adopted to estimate the importance of data points in K clusters. The importance exhibits both intra-cluster and inter-cluster relations of a data point, enabling our method to distinguish both globular and non-globular clusters. It can also reduce the negative effect of noisy points whose importance tends to be a small value. The experimental results show that our proposed approach outperforms several well-known clustering approach across seven complex and non-isotropic datasets. We also evaluate the effectiveness of our algorithm on two real-world datasets, i.e. a public dataset of digit handwriting recognition and a dataset for race walking recognition collected by ourselves, and find our approach outperforms other existing algorithms in most aspects.

A tree-based incremental overlapping clustering method using the three-way decision theory

Existing clustering approaches are usually restricted to crisp clustering, where objects just belong to one cluster; meanwhile there are some applications where objects could belong to more than one cluster. In addition, existing clustering approaches usually analyze static datasets in which objects are kept unchanged after being processed; however many practical datasets are dynamically modified which means some previously learned patterns have to be updated accordingly. In this paper, we propose a new tree-based incremental overlapping clustering method using the three-way decision theory. The tree is constructed from representative points introduced by this paper, which can enhance the relevance of the search result. The overlapping cluster is represented by the three-way decision with interval sets, and the three-way decision strategies are designed to updating the clustering when the data increases. Furthermore, the proposed method can determine the number of clusters during the processing. The experimental results show that it can identifies clusters of arbitrary shapes and does not sacrifice the computing time, and more results of comparison experiments show that the performance of proposed method is better than the compared algorithms in most of cases.

Maximin Separation Probability Clustering

This paper proposes a new approach for discriminative clustering. The intuition is, for a good clustering, one should be able to learn a classifier from the clustering labels with high generalization accuracy. Thus we define a novel metric to evaluate the quality of a clustering labeling, named Minimum Separation Probability (MSP), which is a lower bound of the generalization accuracy of a classifier learnt from the clustering labeling. We take MSP as the objective to maximize and propose our approach Maximin Separation Probability Clustering (MSPC), which has several attractive properties, such as invariance to anisotropic feature scaling and intuitive probabilistic explanation for clustering quality. We present three efficient optimization strategies for MSPC, and analyze their interesting connections to existing clustering approaches, such as maximum margin clustering (MMC) and discriminative k-means. Empirical results on real world data sets verify that MSP is a robust and effective clustering quality measure. It is also shown that the proposed algorithms compare favorably to state-of-the-art clustering algorithms in both accuracy and efficiency.

Cycle-flow–based module detection in directed recurrence networks

We present a new cycle-flow–based method for finding fuzzy partitions of weighted directed networks coming from time series data. We show that this method overcomes essential problems of most existing clustering approaches, which tend to ignore important directional information by considering only one-step, one-directional node connections. Our method introduces a novel measure of communication between nodes using multi-step, bidirectional transitions encoded by a cycle decomposition of the probability flow. Symmetric properties of this measure enable us to construct an undirected graph that captures the information flow of the original graph seen by the data and apply clustering methods designed for undirected graphs. Finally, we demonstrate our algorithm by analyzing earthquake time series data, which naturally induce (time-)directed networks.

Design and Development of Energy Efficient Routing Protocol for Wireless Sensor Networks using Fuzzy Logic

Wireless Sensor Networks (WSNs) is a emerging technology of real time embedded systems for a variety of applications. In general, WSNs has great challenges in the factor of limited computation, energy and memory resources. Clustering techniques play a vital role in WSNs to increase the network lifetime and also made energy efficiency. Existing clustering approaches like LEACH uses neighboring information of the nodes for selecting cluster heads and other nodes spent more energy for transmitting data to cluster head. It was not considered the expected residual energy for selecting a cluster head. In this study, Genetic Algorithm (GA) is used to form optimal clusters based on fitness parameters including Cluster Distance (CD), Direct Distance to Base Station (DDBS) and Energy of nodes. Also, fuzzy logic approach is applied to select optimal cluster head by using expected residual energy that increases the network lifetime. The aim of the study is providing a solution for unbalanced energy consumption problem in a WSN. The simulation results show that the proposed protocol performs well than other protocols like LEACH and LEACH_ERE.

A combinatorial approach for analyzing intra-tumor heterogeneity from high-throughput sequencing data.

Motivation: High-throughput sequencing of tumor samples has shown that most tumors exhibit extensive intra-tumor heterogeneity, with multiple subpopulations of tumor cells containing different somatic mutations. Recent studies have quantified this intra-tumor heterogeneity by clustering mutations into subpopulations according to the observed counts of DNA sequencing reads containing the variant allele. However, these clustering approaches do not consider that the population frequencies of different tumor subpopulations are correlated by their shared ancestry in the same population of cells.Results: We introduce the binary tree partition (BTP), a novel combinatorial formulation of the problem of constructing the subpopulations of tumor cells from the variant allele frequencies of somatic mutations. We show that finding a BTP is an NP-complete problem; derive an approximation algorithm for an optimization version of the problem; and present a recursive algorithm to find a BTP with errors in the input. We show that the resulting algorithm outperforms existing clustering approaches on simulated and real sequencing data.Availability and implementation: Python and MATLAB implementations of our method are available at http://compbio.cs.brown.edu/software/Contact: braphael@cs.brown.eduSupplementary information: Supplementary data are available at Bioinformatics online.

A Comparative Analysis of Different Clustering Approaches for Software Process Improvement

Software development team tries to increase the software quality by decreasing the number of defects as much as possible. Number of defects remaining in a system provides an insight into the quality of the system. Software defects are one of the major factors that can decide the time of software delivery. The proposed system will analyze and categorize the software defects using some cluste ring approach and then the software defects will be measured in each clustered separately. Clustering is the process to present the data in an effective and organized way. There are number of existing clustering approaches but most of them suffer with problem of data distribution. If the distribution is non linear it gives impurities in clustering process. The proposed work is about to use defect prevention for process improvement with the help of clustering algorithms.