Existing Clustering Algorithms Research Articles

Linear clustering process on networks: a comparative study

Abstract By a thorough performance comparison, we compare the recently proposed, operator-based linear clustering process on a network with classical, existing clustering algorithms. The linear clustering process produces clusters or partitions based on the eigenstructure of a linear operator on a graph that replaces nodes to ‘more natural’ positions by attractive and repulsive forces. Synthetic benchmarks, along with real-world networks possessing or lacking a known community structure, are considered. Our comparative analysis demonstrates that our linear clustering process generates superior partitions compared to the algorithms assessed in most instances, while of comparable computational complexity with the simplest existing clustering algorithms.

Multi-agent Reinforcement Learning Clustering Algorithm Based on Silhouette Coefficient

As an important branch of emerging artificial intelligence algorithms, multi-agent reinforcement learning (MARL) has shown strong performance in collaborative environments. It can utilize multiple agents to find the optimal set of strategies for solving sequential decision problem through trial-and-error. One of the main challenges facing multi-agent system is the non-stationarity problem, which brings poor convergence and seriously affects its performance. Clustering is a commonly used unsupervised analytical method in machine learning, which aims to group samples with similar internal properties into the same cluster. In this paper, we propose a MARL clustering algorithm based on silhouette coefficient (SC-MARLC), and use the trial-and-error strategy to find the best cluster groups. In SC-MARLC, we establish a mapping relationship between multi-agent and samples, construct a novel clustering model based on MARL, and design a good clustering subset structure based on the sample silhouette coefficient. The designed structure is helpful for multi-agent system to solve the non-stationary problem. Finally, we compare the performance of SC-MARLC with 11 existing clustering algorithms on fifteen public datasets. The results show that the new clustering algorithm performs best on ten datasets.

Non-parameter clustering algorithm based on chain propagation and natural neighbor

Clustering analysis is a powerful tool for discovering potential knowledge in datasets. However, numerous existing clustering algorithms suffer from heavy reliance on parameter settings and cannot cluster complex manifold data very well. Moreover, although non-parameter clustering algorithms aim to lower the usage threshold, their actual performance in clustering is often unsatisfactory. Addressing how to achieve similar clustering effects for numerous data points with only a few core data points during clustering is a valuable consideration. To alleviate these challenges, a non-parameter clustering algorithm is proposed and named Non-parameter Clustering Algorithm based on Chain Propagation and Natural Neighbor (NPCCPN) in this paper, by jointly using chain propagation and natural neighbor. Specifically, NPCCPN identifies core points through chain propagation and clusters them using the saturated neighborhood graph. This makes the core data extraction and clustering process efficient and non-parameter. Finally, the performance of the method is validated on 15 complex synthetic datasets and 10 real datasets from public UCI database. The experimental results show that the effectiveness and superiority of the proposed algorithm. Purity scores first. ACC scores second without parameters, but the score first algorithm needs to set the parameters.

Fault diagnosis method for bogie bearings of high-speed trains based on weighted-sparse subspace clustering

The bogie bearings are crucial components of the high-speed train transmission system, and any failure of them can severely impact the normal functioning of the vehicle. Nowadays, sparse subspace clustering (SSC) is an effective technology for diagnosing mechanical system faults. However, SSC is easily affected by noise. To address this issue, we introduce the weighted-SSC algorithm, which incorporates weighting coefficients to enhance the connections between similar sample points. Our approach involves extracting the fault characteristic parameters of the vibration signal through wavelet packet transform and singular value decomposition. Subsequently, these parameters are employed to construct corresponding weighted-sparse subspaces. We also examined the selection of a hyperparameter in the weighted coefficients. The resultant sparse representation vectors are then harnessed for the purpose of transductive semi-supervised learning clustering and diagnosing the specific type of fault. To validate the effectiveness of our proposed method, we designed and built a bearing experimental platform. We compare our method with existing clustering algorithms, including K-means, SSC, dimension reduction graph-based SSC, and ant colony optimization-K-means clustering algorithms. The results demonstrate that our proposed weighted-SSC method achieves higher accuracy in fault diagnosis than the other clustering algorithms.

Developed Clustering Algorithms for Engineering Applications: A Review

Clustering algorithms play a pivotal role in the field of engineering, offering valuable insights into complex datasets. This review paper explores the landscape of developed clustering algorithms with a focus on their applications in engineering. The introduction provides context for the significance of clustering algorithms, setting the stage for an in-depth exploration. The overview section delineates fundamental clustering concepts and elucidates the workings of these algorithms. Categorization of clustering algorithms into partitional, hierarchical, and density-based forms lay the groundwork for a comprehensive discussion. The core of the paper delves into an extensive review of clustering algorithms tailored for engineering applications. Each algorithm is scrutinized in dedicated subsections, unraveling their specific contributions, applications, and advantages. A comparative analysis assesses the performance of these algorithms, delineating their strengths and limitations. Trends and advancements in the realm of clustering algorithms for engineering applications are thoroughly examined. The review concludes with a reflection on the challenges faced by existing clustering algorithms and proposes avenues for future research. This paper aims to provide a valuable resource for researchers, engineers, and practitioners, guiding them in the selection and application of clustering algorithms for diverse engineering scenarios.

DEMOS: Clustering by Pruning a Density-Boosting Cluster Tree of Density Mounts

Most existing clustering algorithms require presetting cluster number and often fail to capture complex shapes. Herein, we propose a clustering algorithm by pruning a density-boosting cluster tree of density mounts—DEnsity MOuntains Separation clustering algorithm (DEMOS). A cluster is assumed to be a density-connected area with multiple (or a single) density mounts (i.e., single-peak clusters) and a relatively large dis-connectivity from density-connected areas of higher densities. Based on this assumption, DEMOS can easily detect the number of clusters and robustly reconstruct their complex shapes. It first builds the dataset into a peak graph, where each density peak represents a density mount. A multi-valley-link-based connectivity estimation method is embedded to efficiently estimate the connectivity between density peaks during peak graph building. Then, by applying a new linkage metric designed based on our assumption, DEMOS builds density mounts into a reasonably density-boosting cluster tree. After obtaining a robust center detection in a clarity-enhancing decision graph (i.e., a two-dimensional plot for detecting centers), DEMOS prunes the cluster tree into final clusters to finish clustering. Experimental results on both synthetic and real datasets demonstrated the effectiveness of DEMOS and its applicability to large-scale data clustering.

CGUFS: A clustering-guided unsupervised feature selection algorithm for gene expression data

(Aim) Gene expression data is typically high dimensional with a limited number of samples and contain many features that are unrelated to the disease of interest. Existing unsupervised feature selection algorithms primarily focus on the significance of features in maintaining the data structure while not taking into account the redundancy among features. Determining the appropriate number of significant features is another challenge. (Method) In this paper, we propose a clustering-guided unsupervised feature selection (CGUFS) algorithm for gene expression data that addresses these problems. Our proposed algorithm introduces three improvements over existing algorithms. For the problem that existing clustering algorithms require artificially specifying the number of clusters, we propose an adaptive k-value strategy to assign appropriate pseudo-labels to each sample by iteratively updating a change function. For the problem that existing algorithms fail to consider the redundancy among features, we propose a feature grouping strategy to group highly redundant features. For the problem that the existing algorithms cannot filter the redundant features, we propose an adaptive filtering strategy to determine the feature combinations to be retained by calculating the potentially effective features and potentially redundant features of each feature group. (Result) Experimental results show that the average accuracy (ACC) and matthews correlation coefficient (MCC) indexes of the C4.5 classifier on the optimal features selected by the CGUFS algorithm reach 74.37% and 63.84%, respectively, significantly superior to the existing algorithms. (Conclusion) Similarly, the average ACC and MCC indexes of the Adaboost classifier on the optimal features selected by the CGUFS algorithm are significantly superior to the existing algorithms. In addition, statistical experiment results show significant differences between the CGUFS algorithm and the existing algorithms.

Application of Semi-supervised Fuzzy Clustering Based on Knowledge Weighting and Cluster Center Learning to Mammary Molybdenum Target Image Segmentation.

Breast cancer is commonly diagnosed with mammography. Using image segmentation algorithms to separate lesion areas in mammography can facilitate diagnosis by doctors and reduce their workload, which has important clinical significance. Because large, accurately labeled medical image datasets are difficult to obtain, traditional clustering algorithms are widely used in medical image segmentation as an unsupervised model. Traditional unsupervised clustering algorithms have limited learning knowledge. Moreover, some semi-supervised fuzzy clustering algorithms cannot fully mine the information of labeled samples, which results in insufficient supervision. When faced with complex mammography images, the above algorithms cannot accurately segment lesion areas. To address this, a semi-supervised fuzzy clustering based on knowledge weighting and cluster center learning (WSFCM_V) is presented. According to prior knowledge, three learning modes are proposed: a knowledge weighting method for cluster centers, Euclidean distance weights for unlabeled samples, and learning from the cluster centers of labeled sample sets. These strategies improve the clustering performance. On real breast molybdenum target images, the WSFCM_V algorithm is compared with currently popular semi-supervised and unsupervised clustering algorithms. WSFCM_V has the best evaluation index values. Experimental results demonstrate that compared with the existing clustering algorithms, WSFCM_V has a higher segmentation accuracy than other clustering algorithms, both for larger lesion regions like tumor areas and for smaller lesion areas like calcification point areas.

Segmentation of breast molybdenum target image lesions based on semi-supervised fuzzy clustering

Currently, breast cancer is one of the most common cancers among women. To aid clinicians in diagnosis, lesion regions in mammography pictures can be segmented using an artificial intelligence system. This has significant clinical implications. Clustering algorithms, as unsupervised models, are widely used in medical image segmentation. However, due to the different sizes and shapes of lesions in mammography images and the low contrast between lesion areas and the surrounding pixels, it is difficult to use traditional unsupervised clustering methods for image segmentation. In this study, we try to apply the semisupervised fuzzy clustering algorithm to lesion segmentation in mammography molybdenum target images and propose semisupervised fuzzy clustering based on the cluster centres of labelled samples (called SFCM_V, where V stands for cluster centre). The algorithm refers to the cluster centre of the labelled sample dataset during the clustering process and uses the information of the labelled samples to guide the unlabelled samples during clustering to improve the clustering performance. We compare the SFCM_V algorithm with the current popular semisupervised clustering algorithm and an unsupervised clustering algorithm and perform experiments on real patient mammogram images using DICE and IoU as evaluation metrics; SFCM_V has the highest evaluation metric coefficient. Experiments demonstrate that SFCM_V has higher segmentation accuracy not only for larger lesion regions, such as tumours, but also for smaller lesion regions, such as calcified spots, compared with existing clustering algorithms.

A Distributed Clustering Scheme for Dense Wireless Sensor Networks

Abstract: A sensor node, also known as an Alfeo (mainly in North America), is a sensor network node capable of processing, collecting sensor data, and communicating with other connected nodes in the network. Alpheus is a node, but a node is not always a point. Each sensor node has a microprocessor and some memory. In addition, each sensor node has one or more sensor devices such as acoustic sensors, microphone arrays, and video cameras, infrared, seismic or magnetic sensors. However, dead batteries in sensor nodes are difficult to replace. A typical sensor node consumes part of its energy during wireless communication. This research paper proposes to develop a well estimated distributed clustering model for dense wireless sensor network fields that provides improved performance over the existing clustering algorithm LEACH. The two robbery concepts behind the proposed system are the hierarchical distributed clustering mechanism and the threshold concept. Energy consumption is significantly reduced, which significantly extends the life of sensor nodes

A New Sparse Data Clustering Method Based On Frequent Items

Large, sparse categorical data is a natural way to represent complex data like sequences, trees, and graphs. Such data is prevalent in many applications, e.g., Criteo released a terabyte size click log data of 4 billion records with millions of dimensions. While most existing clustering algorithms like k-Means work well on dense, numerical data, there exist relatively few algorithms that can cluster sets of sparse categorical features. In this paper, we propose a new method called k-FreqItems that performs scalable clustering over high-dimensional, sparse data. To make clustering results easily interpretable, k-FreqItems is built upon a novel sparse center representation called FreqItem which will choose a set of high-frequency, non-zero dimensions to represent the cluster. Unlike most existing clustering algorithms, which adopt Euclidean distance as the similarity measure, k-FreqItems uses the popular Jaccard distance for comparing sets. Since the efficiency and effectiveness of k-FreqItems are highly dependent on an initial set of representative seeds, we introduce a new randomized initialization method, SILK, to deal with the seeding problem of k-FreqItems. SILK uses locality-sensitive hash (LSH) functions for oversampling and identifies frequently co-occurred data in LSH buckets to determine a set of promising seeds, allowing k-FreqItems to converge swiftly in an iterative process. Experimental results over seven real-world sparse data sets show that the SILK seeding is around 1.1\sim3.2× faster yet more effective than the state-of-the-art seeding methods. Notably, SILK scales up well to a billion data objects on a commodity machine with 4 GPUs. The code is available at https://github.com/HuangQiang/k-FreqItems.

An ensemble method for estimating the number of clusters in a big data set using multiple random samples

Clustering a big dataset without knowing the number of clusters presents a big challenge to many existing clustering algorithms. In this paper, we propose a Random Sample Partition-based Centers Ensemble (RSPCE) algorithm to identify the number of clusters in a big dataset. In this algorithm, a set of disjoint random samples is selected from the big dataset, and the I-niceDP algorithm is used to identify the number of clusters and initial centers in each sample. Subsequently, a cluster ball model is proposed to merge two clusters in the random samples that are likely sampled from the same cluster in the big dataset. Finally, based on the ball model, the RSPCE ensemble method is used to ensemble the results of all samples into the final result as a set of initial cluster centers in the big dataset. Intensive experiments were conducted on both synthetic and real datasets to validate the feasibility and effectiveness of the proposed RSPCE algorithm. The experimental results show that the ensemble result from multiple random samples is a reliable approximation of the actual number of clusters, and the RSPCE algorithm is scalable to big data.

Spectral Clustering With Adaptive Neighbors for Deep Learning.

Spectral clustering is a well-known clustering algorithm for unsupervised learning, and its improved algorithms have been successfully adapted for many real-world applications. However, traditional spectral clustering algorithms are still facing many challenges to the task of unsupervised learning for large-scale datasets because of the complexity and cost of affinity matrix construction and the eigen-decomposition of the Laplacian matrix. From this perspective, we are looking forward to finding a more efficient and effective way by adaptive neighbor assignments for affinity matrix construction to address the above limitation of spectral clustering. It tries to learn an affinity matrix from the view of global data distribution. Meanwhile, we propose a deep learning framework with fully connected layers to learn a mapping function for the purpose of replacing the traditional eigen-decomposition of the Laplacian matrix. Extensive experimental results have illustrated the competitiveness of the proposed algorithm. It is significantly superior to the existing clustering algorithms in the experiments of both toy datasets and real-world datasets.

An automated robust algorithm for clustering multivariate data

Clustering analysis is widely used in various applications, such as marketing, biology, medical science, finance, data mining, image processing, data analysis and pattern recognition. For instance, clustering can be used to: characterize the customer groups based on their purchasing patterns by discovering the distinct groups in the customer base; derive plant and animal taxonomies, categorize genes with similar functionalities and gain insight into structures inherent to populations; identify the cancer cells; and detect credit card fraud. The k-means, Hierarchical and self-organizing (Kohonen) map are widely used clustering algorithms. The practice demonstrated that these clustering algorithms have some significant limitations and drawbacks. This manuscript gives an automated robust algorithm for clustering multivariate data without prior information about the number of clusters. Robust estimate of location and covariance matrix are used to define Mahalanobis distance and corresponding radius of clustering algorithm. The algorithm is designed in a way that it controls both masking and swamping effects. It automatically divides a given data set into a number of clusters. Some properties pertaining to the algorithm are demonstrated which helps in finding clusters that accommodates observations with large deviation. A method to avoid the use of a fixed cutoff for determining outlier is discussed. The performance of the proposed algorithm is compared with the existing clustering algorithms and robust multiple outlier detection methods.

Variational kinetic clustering of complex networks.

Efficiently identifying the most important communities and key transition nodes in weighted and unweighted networks is a prevalent problem in a wide range of disciplines. Here, we focus on the optimal clustering using variational kinetic parameters, linked to Markov processes defined on the underlying networks, namely, the slowest relaxation time and the Kemeny constant. We derive novel relations in terms of mean first passage times for optimizing clustering via the Kemeny constant and show that the optimal clustering boundaries have equal round-trip times to the clusters they separate. We also propose an efficient method that first projects the network nodes onto a 1D reaction coordinate and subsequently performs a variational boundary search using a parallel tempering algorithm, where the variational kinetic parameters act as an energy function to be extremized. We find that maximization of the Kemeny constant is effective in detecting communities, while the slowest relaxation time allows for detection of transition nodes. We demonstrate the validity of our method on several test systems, including synthetic networks generated from the stochastic block model and real world networks (Santa Fe Institute collaboration network, a network of co-purchased political books, and a street network of multiple cities in Luxembourg). Our approach is compared with existing clustering algorithms based on modularity and the robust Perron cluster analysis, and the identified transition nodes are compared with different notions of node centrality.

An Enhanced Expectation Maximization Text Document Clustering Algorithm for E-Content Analysis

Nowadays, there are many types of digital materials that can be used in the classroom. Students and scholars are migrating from textbooks to digital study materials because textbooks are too large and expensive. Teachers and college students can use and modify the materials that are freely available or with some constraints for their learning and teaching. E-content can be designed, evolved, utilized, re-used, and distributed electronically from anywhere at anytime. Because of the flexibility of time, place, and speed of learning, e-content is becoming extremely popular. It can be readily and instantly shared and communicated with an infinite number of clients all across the globe. Document clustering is most commonly used to group documents that are related to a specific topic. Text document clustering can be used to group a collection of documents regarding the information they include and to deliver search results when a user searches the internet. In this paper mainly focuses on text document clustering to cope with massive collection of E-Content documents. Enhanced Expectation Maximization Text Document Clustering (EEMTDC) clustering algorithm was proposed and compared with Expectation Maximization (EM) clustering, K-Means clustering, and Hierarchical clustering (HC) algorithms. The experiment shows that the performance of proposed EEMTDC algorithm produces greater clustering accuracy than existing clustering algorithms.

DDCAL: Evenly Distributing Data into Low Variance Clusters Based on Iterative Feature Scaling

This work studies the problem of clustering one-dimensional data points such that they are evenly distributed over a given number of low variance clusters. One application is the visualization of data on choropleth maps or on business process models, but without over-emphasizing outliers. This enables the detection and differentiation of smaller clusters. The problem is tackled based on a heuristic algorithm called DDCAL (1d distribution cluster algorithm) that is based on iterative feature scaling which generates stable results of clusters. The effectiveness of the DDCAL algorithm is shown based on 5 artificial data sets with different distributions and 4 real-world data sets reflecting different use cases. Moreover, the results from DDCAL, by using these data sets, are compared to 11 existing clustering algorithms. The application of the DDCAL algorithm is illustrated through the visualization of pandemic and population data on choropleth maps as well as process mining results on process models.

A generalized Bayes framework for probabilistic clustering

Summary Loss-based clustering methods, such as k-means clustering and its variants, are standard tools for finding groups in data. However, the lack of quantification of uncertainty in the estimated clusters is a disadvantage. Model-based clustering based on mixture models provides an alternative approach, but such methods face computational problems and are highly sensitive to the choice of kernel. In this article we propose a generalized Bayes framework that bridges between these paradigms through the use of Gibbs posteriors. In conducting Bayesian updating, the loglikelihood is replaced by a loss function for clustering, leading to a rich family of clustering methods. The Gibbs posterior represents a coherent updating of Bayesian beliefs without needing to specify a likelihood for the data, and can be used for characterizing uncertainty in clustering. We consider losses based on Bregman divergence and pairwise similarities, and develop efficient deterministic algorithms for point estimation along with sampling algorithms for uncertainty quantification. Several existing clustering algorithms, including k-means, can be interpreted as generalized Bayes estimators in our framework, and thus we provide a method of uncertainty quantification for these approaches, allowing, for example, calculation of the probability that a data point is well clustered.

Deep Embedded Clustering Framework for Mixed Data

Deep embedded clustering (DEC) is a representative clustering algorithm that leverages deep-learning frameworks. DEC jointly learns low-dimensional feature representations and optimizes the clustering goals but only works with numerical data. However, in practice, the real-world data to be clustered includes not only numerical features but also categorical features that DEC cannot handle. In addition, if the difference between the soft assignment and target values is large, DEC applications may suffer from convergence problems. In this study, to overcome these limitations, we propose a deep embedded clustering framework that can utilize mixed data to increase the convergence stability using soft-target updates; a concept that is borrowed from an improved deep Q learning algorithm used in reinforcement learning. To evaluate the performance of the framework, we utilized various benchmark datasets composed of mixed data and empirically demonstrated that our approach outperformed existing clustering algorithms in most standard metrics. To the best of our knowledge, we state that our work achieved state-of-the-art performance among its contemporaries in this field.

이상치를 고려한 딥러닝 기반 웨이퍼 맵 결함 패턴 클러스터링

Purpose: Most clustering algorithms perform clustering without considering outliers. However, when performing clustering, outliers often degrade performance. Therefore, we propose a clustering method that can consider outliers.BRMethods: The proposed deep learning-based algorithm identifies outliers using the predicted class distribution and performs clustering based on only normal data.BRResults: The proposed algorithm achieved superior performance compared to the existing clustering algorithm. Based on the proposed clustering algorithm, the wafer maps were clustered while finding outliers in the patterns existing on them.BRConclusion: In this paper, we propose a deep learning-based clustering algorithm considering outliers (DCCO) that simultaneously performs clustering and abnormal pattern detection.