Traditional Clustering Methods Research Articles

Preserving Global Information for Graph Clustering with Masked Autoencoders

Graph clustering aims to divide nodes into different clusters without labels and has attracted great attention due to the success of graph neural networks (GNNs). Traditional GNN-based clustering methods are based on the homophilic assumption, i.e., connected nodes belong to the same clusters. However, this assumption is not always true, as heterophilic graphs are also ubiquitous in the real world, which limits the application of GNNs. Furthermore, these methods overlook global positions, which can result in erroneous clustering. To solve the aforementioned problems, we propose a novel model called Preserving Global Information for Graph Clustering with Masked Autoencoders (GCMA). We first propose a low–high-pass filter to capture meaningful low- and high-frequency information. Then, we propose a graph diffusion method to obtain the global position. Specifically, a parameterized Laplacian matrix is proposed to better control the global direction. To further enhance the learning ability of the autoencoders, we design a model with a masking strategy that enhances the learning ability. Extensive experiments on both homophilic and heterophilic graphs demonstrate GCMA’s advantages over state-of-the-art baselines.

Multitask-Guided Deep Clustering With Boundary Adaptation.

Multitask learning uses external knowledge to improve internal clustering and single-task learning. Existing multitask learning algorithms mostly use shallow-level correlation to aid judgment, and the boundary factors on high-dimensional datasets often lead algorithms to poor performance. The initial parameters of these algorithms cause the border samples to fall into a local optimal solution. In this study, a multitask-guided deep clustering (DC) with boundary adaptation (MTDC-BA) based on a convolutional neural network autoencoder (CNN-AE) is proposed. In the first stage, dubbed multitask pretraining (M-train), we construct an autoencoder (AE) named CNN-AE using the DenseNet-like structure, which performs deep feature extraction and stores captured multitask knowledge into model parameters. In the second phase, the parameters of the M-train are shared for CNN-AE, and clustering results are obtained by deep features, which is termed as single-task fitting (S-fit). To eliminate the boundary effect, we use data augmentation and improved self-paced learning to construct the boundary adaptation. We integrate boundary adaptors into the M-train and S-fit stages appropriately. The interpretability of MTDC-BA is accomplished by data transformation. The model relies on the principle that features become important as the reconfiguration loss decreases. Experiments on a series of typical datasets confirm the performance of the proposed MTDC-BA. Compared with other traditional clustering methods, including single-task DC algorithms and the latest multitask clustering algorithms, our MTDC-BA achieves better clustering performance with higher computational efficiency. Deep features clustering results demonstrate the stability of MTDC-BA by visualization and convergence verification. Through the visualization experiment, we explain and analyze the whole model data input and the middle characteristic layer. Further understanding of the principle of MTDC-BA. Through additional experiments, we know that the proposed MTDC-BA is efficient in the use of multitask knowledge. Finally, we carry out sensitivity experiments on the hyper-parameters to verify their optimal performance.

Cauchy hyper-graph Laplacian nonnegative matrix factorization for single-cell RNA-sequencing data analysis.

Many important biological facts have been found as single-cell RNA sequencing (scRNA-seq) technology has advanced. With the use of this technology, it is now possible to investigate the connections among individual cells, genes, and illnesses. For the analysis of single-cell data, clustering is frequently used. Nevertheless, biological data usually contain a large amount of noise data, and traditional clustering methods are sensitive to noise. However, acquiring higher-order spatial information from the data alone is insufficient. As a result, getting trustworthy clustering findings is challenging. We propose the Cauchy hyper-graph Laplacian non-negative matrix factorization (CHLNMF) as a unique approach to address these issues. In CHLNMF, we replace the measurement based on Euclidean distance in the conventional non-negative matrix factorization (NMF), which can lessen the influence of noise, with the Cauchy loss function (CLF). The model also incorporates the hyper-graph constraint, which takes into account the high-order link among the samples. The CHLNMF model's best solution is then discovered using a half-quadratic optimization approach. Finally, using seven scRNA-seq datasets, we contrast the CHLNMF technique with the other nine top methods. The validity of our technique was established by analysis of the experimental outcomes.

Global aerosol-type classification using a new hybrid algorithm and Aerosol Robotic Network data

Abstract. The properties of aerosols are highly uncertain owing to the complex changes in their composition in different regions. The radiative properties of different aerosol types differ considerably and are vital for studying aerosol regional and/or global climate effects. Traditional aerosol-type identification algorithms, generally based on cluster or empirical analysis methods, are often inaccurate and time-consuming. In response, our study aimed to develop a new aerosol-type classification model using an innovative hybrid algorithm to improve the precision and efficiency of aerosol-type identification. This novel algorithm incorporates an optical database, constructed using the Mie scattering model, and employs a random forest algorithm to classify different aerosol types based on the optical data from the database. The complex refractive index was used as a baseline to assess the performance of our hybrid algorithm against the traditional Gaussian kernel density clustering method for aerosol-type identification. The hybrid algorithm demonstrated impressive consistency rates of 90 %, 85 %, 84 %, 84 %, and 100 % for dust, mixed-coarse (mixed, course-mode aerosol), mixed-fine (mixed, fine-mode aerosol), urban/industrial, and biomass burning aerosols, respectively. Moreover, it achieved remarkable precision, with evaluation metric indexes for micro-precision, micro-recall, micro-F1-score, and accuracy of 95 %, 89 %, 91 %, and 89 %, respectively. Lastly, a global map of aerosol types was generated using the new hybrid algorithm to characterize aerosol types across the five continents. This study, utilizing a novel approach for the classification of aerosol, will help improve the accuracy of aerosol inversion and determine the sources of aerosol pollution.

Multivariate time series clustering based on fuzzy cognitive maps and community detection

Most time series clustering methods mainly focus on univariate time series (UTS). Compared with UTS, multivariate time series (MTS) consists of multiple components. Although interest in MTS clustering is increasing, its performance is far from satisfactory. Most traditional MTS clustering methods may have two limitations. First, they do not consider both the temporal features of each component and the relationship between the components. Second, they can only identify the local relationship between adjacent data, but cannot obtain long-distance global relationships and capture clusters of arbitrary shapes. In this paper, we develop a method for MTS clustering based on fuzzy cognitive maps (FCMs) and community detection, termed as MTSC-FCM-CD. To overcome the first limitation, we use FCM to represent MTS; FCM can extract temporal features while preserving the relationship between components in MTS. To overcome the second limitation, we use the community detection algorithm to cluster the global relations, which is different from the traditional nearest neighbor distance-based method. In the calculation process, the similarity between FCMs should be used to build a complex network. Existing methods calculate the similarity between FCMs, only considering the numerical characteristics but ignoring the topological structure. To solve this problem, we design a two-stage similarity measure to build a complex network. In comparison to the existing methods, the experimental results on twenty benchmark datasets demonstrate the effectiveness of MTSC-FCM-CD in MTS clustering.

Survey on graph neural network-based community detection and its applications

Detecting communities within a network is a critical component of network analysis. The process involves identifying clusters of nodes that exhibit greater similarity to each other compared to other nodes in the network. In the context of Complex networks (CN), community detection becomes even more important as these clusters provide relevant information of interest. Traditional mathematical and clustering methods have limitations in terms of data visualization and high-dimensional information extraction. To address these challenges, graph neural network learning methods have gained popularity in community detection, as they are capable of handling complex structures and multi-dimensional data. Developing a framework for community detection in complex networks using graph neural network learning is a challenging and ongoing research objective. Therefore, it is essential for researchers to conduct a thorough review of community detection techniques that utilize cutting-edge graph neural network learning methods [102], in order to analyze and construct effective detection models. This paper provides a brief overview of graph neural network learning methods based on community detection methods and summarizes datasets, evaluation metrics, applications, and challenges of community detection in complex networks.

SurvivalLVQ: Interpretable supervised clustering and prediction in survival analysis via Learning Vector Quantization

Identifying subgroups with similar survival outcomes is a pivotal challenge in survival analysis. Traditional clustering methods often neglect the outcome variable, potentially leading to inaccurate representation of risk profiles. To address this, we present SurvivalLVQ, a novel interpretable method that adapts Learning Vector Quantization (LVQ) to survival analysis. Unlike traditional classification uses of LVQ, SurvivalLVQ groups individuals by survival probabilities and assigns a unique survival curve to each cluster, representing the collective survival behavior within that group. Moreover, it can predict individual survival curves using weighted averages from nearby clusters. When tested on 76 benchmark datasets, it outperformed other clustering methods and showed competitive prediction performance. SurvivalLVQ bridges the gap between clustering techniques and outcome-oriented methods. Its strong clustering performance, coupled with competitive prediction capabilities and with easy to interpret outcomes, make it a promising tool for various applications within survival analysis.

Automated litho-fluid and facies classification in well-logs: the Rock Physics perspective

While accurate litho-fluid and facies interpretation from wireline log data is critical for applications like joint facies and impedance inversion of seismic data, extracting this information manually is challenging due to the complexity and high dimensionality of the logs. Traditional clustering methods also struggle with litho-fluid type inference due to different depth trends in petrophysical rock properties due to compaction and diagenesis. We introduce a Rock Physics Machine Learning workflow that automates litho-fluid classification and property depth trend modeling to address these challenges. This workflow employs a maximum-likelihood approach, explicitly accounting for depth-related effects via Rock Physics models, to infer litho-fluid types from borehole data. It utilizes a robust Expectation-Maximization algorithm to associate each litho-fluid type with a specific Rock Physics model instance, constrained within physically reasonable bounds. The workflow directly outputs litho-fluid type proportions and type-specific Rock Physics models with associated uncertainties, providing essential prior information for seismic inversion.

Fast Clustering With Anchor Guidance.

Clustering aims to partition a set of objects into different groups through the internal nature of these objects. Most existing methods face intractable hyper-parameter problems triggered by various regularization terms, which degenerates the applicability of models. Moreover, traditional graph clustering methods always encounter the expensive time overhead. To this end, we propose a Fast Clustering model with Anchor Guidance (FCAG). The proposed model not only avoids trivial solutions without extra regularization terms, but is also suitable to deal with large-scale problems by utilizing the prior knowledge of the bipartite graph. Moreover, the proposed FCAG can cope with out-of-sample extension problems. Three optimization methods Projected Gradient Descent (PGD) method, Iteratively Re-Weighted (IRW) algorithm and Coordinate Descent (CD) algorithm are proposed to solve FCAG. Extensive experiments verify the superiority of the optimization method CD. Besides, compared with other bipartite graph models, FCAG has the better performance with the less time cost. In addition, we prove through theory and experiment that when the learning rate of PGD tends to infinite, PGD is equivalent to IRW.

Flexible multi-objective particle swarm optimization clustering with game theory to address human activity discovery fully unsupervised

Human activity recognition is a crucial field of study, but current approaches often require ground truth labels, which are not always available. We propose a new method called the Flexible Multi-Objective Particle swarm optimization clustering method based on Game theory (FMOPG), which can identify human activities without any supervision. Unlike traditional clustering methods that require an estimate of the number of clusters and are often inaccurate, FMOPG handles varying cluster numbers with an incremental technique, selecting clusters with good connectivity and separation. We enhance Particle Swarm Optimization (PSO) with mean-shift vectors for faster convergence and better handling of non-spherical clusters. Employing multi-objective optimization and Gaussian mutation, FMOPG provides a range of optimal solutions. We map the optimization problem to game theory to select the best solution based on different criteria. A smart grid-based method is proposed for population initialization, reducing variance and improving reliability. FMOPG outperforms state-of-the-art methods, improving clustering accuracy by 3.65%. Moreover, the incremental technique has improved clustering time by 71.18%.

Deep Attentional Implanted Graph Clustering Algorithm for the Visualization and Analysis of Social Networks

As the user base expands, social network data becomes more intricate, making analyzing the interconnections between various entities challenging. Various graph visualization technologies are employed to analyze extensive and intricate network data. Network graphs inherently possess intricacy and may have overlapping elements. Graph clustering is a basic endeavor that aims to identify communities or groupings inside networks. Recent research has mostly concentrated on developing deep learning techniques to acquire a concise representation of graphs, which is then utilized with traditional clustering methods such as k-means or spectral clustering techniques. Multiplying these two-step architectures is challenging and sometimes results in unsatisfactory performance. This is mostly due to the lack of a goal-oriented graph encoding developed explicitly for the clustering job. This work introduces a novel Deep Learning (DL) method called Deep Attentional Implanted Graph Clustering (DAIGC), designed to achieve goal-oriented clustering. Our approach centers on associated graphs to thoroughly investigate both aspects of data in graphs. The proposed DAIGC technique utilizes a Graph Attention Autoencoder (GAA) to determine the significance of nearby nodes about a target node. This allows encoding a graph's topographical structure and node value into a concise representation. Based on this representation, an interior product decoder has been trained to rebuild the graph structure. The performance of the proposed approach has been evaluated on four distinct types and sizes of real-world intricate networks, varying in vertex count from 𝑁=102 𝑡𝑜 𝑁=107. The performance of the suggested methods is evaluated by comparing them with two established and commonly used graph clustering techniques. The testing findings demonstrate the effectiveness of the proposed method in terms of processing speed and visualization compared to the state-of-the-art algorithms.

Deep Learning for Clustering Single-cell RNA-seq Data

Abstract: The development of single-cell RNA sequencing (scRNA-seq) technology provides an excellent opportunity to explore cell heterogeneity and diversity. With the growing application of scRNA-seq data, many computational clustering methods have been developed to further uncover cell subgroups, and cell dynamics at the group level. Due to the characteristics of high dimension, high sparsity and high noise of the scRNA-seq data, it is challenging to use traditional clustering methods. Fortunately, deep learning technologies characterize the properties of scRNA-seq data well and provide a new perspective for data analysis. This work reviews the most popular computational clustering methods and tools based on deep learning technologies, involving comparison, data collection, code acquisition, results evaluation, and so on. In general, such a presentation points out some progress and limitations of the existing methods and discusses the challenges and directions for further research, which may give new insight to address a broader range of new challenges in dealing with single-cell sequencing data and downstream analysis.

Hyperlink communities in higher-order networks

Abstract Many networks can be characterized by the presence of communities, which are groups of units that are closely linked. Identifying these communities can be crucial for understanding the system’s overall function. Recently, hypergraphs have emerged as a fundamental tool for modelling systems where interactions are not limited to pairs but may involve an arbitrary number of nodes. In this study, we adopt a dual approach to community detection and extend the concept of link communities to hypergraphs. This extension allows us to extract informative clusters of highly related hyperedges. We analyse the dendrograms obtained by applying hierarchical clustering to distance matrices among hyperedges across a variety of real-world data, showing that hyperlink communities naturally highlight the hierarchical and multiscale structure of higher-order networks. Moreover, hyperlink communities enable us to extract overlapping memberships from nodes, overcoming limitations of traditional hard clustering methods. Finally, we introduce higher-order network cartography as a practical tool for categorizing nodes into different structural roles based on their interaction patterns and community participation. This approach aids in identifying different types of individuals in a variety of real-world social systems. Our work contributes to a better understanding of the structural organization of real-world higher-order systems.

A deep clustering framework for load pattern segmentation

In recent years, the widespread use of smart meters in power networks has generated a wealth of data from electricity customers. However, much of this real-world smart meter dataset lacks labeled data, posing a significant challenge that can be addressed by clustering. Traditional clustering methods often struggle in high-dimensional spaces, leading to less accurate results and increased computational demands. In response to these challenges, this research introduces a framework that utilizes a deep-learning-based clustering approach to address the issue of accurate customer clustering based on usage patterns in unlabeled data. Using an autoencoder, our approach integrates dimensionality reduction and clustering into an end-to-end unsupervised learning framework. Our algorithm significantly improves load profiling by tackling challenges related to nonlinear decision boundaries at the autoencoder bottleneck. Unlike traditional approaches, we propose separating the optimization of reconstruction and cluster loss, bridging the gap between clustering quality and reconstruction efficiency. We rigorously analyze the performance of our approach by comparing classical and state-of-the-art algorithms using two real-world smart meter data. We provide a comprehensive comparative analysis of our method against five common dimension reduction techniques used in high-dimensional clustering. The experimental analysis concludes that the proposed algorithm significantly enhances load profiling more than others, as confirmed through detailed load curve analysis and clustering validity indexes. This comprehensive assessment highlights the effectiveness and versatility of our proposed methodology when compared to others. Moreover, this research advances load profiling in smart grid analytics, providing practical insights for utilities and stakeholders looking to optimize power network operations.

Recognizing and detecting COVID-19 in chest X-ray images using constrained multi-view spectral clustering

AbstractMachine learning, particularly classification algorithms, has been widely employed for diagnosing COVID-19 cases. However, these methods typically rely on labeled datasets and analyze a single data view. With the vast amount of patient data available without labels, this paper addresses the novel challenge of unsupervised COVID-19 diagnosis. The goal is to harness the abundant data without labels effectively. In recent times, multi-view clustering has garnered considerable attention in the research community. Spectral clustering, known for its robust theoretical framework, is a key focus. However, traditional spectral clustering methods generate only nonlinear data projections, necessitating additional clustering steps. The quality of these post-processing steps can be influenced by various factors, such as initialization procedures and outliers. This paper introduces an enhanced version of the recent “Multiview Spectral Clustering via integrating Nonnegative Embedding and Spectral Embedding” method. While retaining the benefits of the original technique, the proposed model integrates two essential constraints: (1) a constraint for ensuring the consistent smoothness of the nonnegative embedding across all views and (2) an orthogonality constraint imposed on the columns of the nonnegative embedding matrix. The effectiveness of this approach is demonstrated using COVIDx datasets. Additionally, the method is evaluated on other image datasets to validate its suitability for this study.

Clustering on hierarchical heterogeneous data with prior pairwise relationships

BackgroundClustering is a fundamental problem in statistics and has broad applications in various areas. Traditional clustering methods treat features equally and ignore the potential structure brought by the characteristic difference of features. Especially in cancer diagnosis and treatment, several types of biological features are collected and analyzed together. Treating these features equally fails to identify the heterogeneity of both data structure and cancer itself, which leads to incompleteness and inefficacy of current anti-cancer therapies.ObjectivesIn this paper, we propose a clustering framework based on hierarchical heterogeneous data with prior pairwise relationships. The proposed clustering method fully characterizes the difference of features and identifies potential hierarchical structure by rough and refined clusters.ResultsThe refined clustering further divides the clusters obtained by the rough clustering into different subtypes. Thus it provides a deeper insight of cancer that can not be detected by existing clustering methods. The proposed method is also flexible with prior information, additional pairwise relationships of samples can be incorporated to help to improve clustering performance. Finally, well-grounded statistical consistency properties of our proposed method are rigorously established, including the accurate estimation of parameters and determination of clustering structures.ConclusionsOur proposed method achieves better clustering performance than other methods in simulation studies, and the clustering accuracy increases with prior information incorporated. Meaningful biological findings are obtained in the analysis of lung adenocarcinoma with clinical imaging data and omics data, showing that hierarchical structure produced by rough and refined clustering is necessary and reasonable.

A novel incomplete hesitant fuzzy information supplement and clustering method for large-scale group decision-making.

Clustering is an effective means to reduce the scaling of large-scale group decision-making (LSGDM). However, there are many problems with clustering methods, such as incomplete or ambiguous information usually provided by different decision makers. Traditional clustering methods may not be able to handle these situations effectively, resulting in incomplete decision-making information. Calculating the clustering centers may become very complex and time-consuming. Inappropriate distance weights may also lead to incorrect cluster assignments, and these problems will seriously affect the clustering results. This research provides a novel incomplete hesitant fuzzy information supplement and clustering approach for large-scale group decision-making in order to address the aforementioned difficulties. First, the approach takes into account the trust degradation and the inhibition of relationships of distrust in the process of trust propagation, and then it builds a global and local network of trust. A novel supplemental formula is provided that takes into account the decision-preference maker's as well as the trust-neighbor's information, allowing the decision-neighbor maker's recommendation to be realized. Therefore, an improved distance function can be proposed to calculate the weights by combining the relative standard deviation theory and selecting the selected clustering centers by using the density peaks in order to optimize the selection of clustering centers and reduce the complexity and scaling of the decision. Finally, an example is presented to demonstrate how the proposed method can be applied. The consistency index and comparison experiments are used to evaluate if the suggested approach is effective and reliable.

Fast Clustering via Maximizing Adaptively Within-Class Similarity.

Clustering aims to make data points in the same group have higher similarity or make data points in different groups have lower similarity. Therefore, we propose three novel fast clustering models motivated by maximizing within-class similarity, which can obtain more instinct clustering structure of data. Different from traditional clustering methods, we divide all n samples into m classes by the pseudo label propagation algorithm first, and then m classes are merged to c classes ( ) by the proposed three co-clustering models, where c is the real number of categories. On the one hand, dividing all samples into more subclasses first can preserve more local information. On the other hand, proposed three co-clustering models are motivated by the thought of maximizing the sum of within-class similarity, which can utilize the dual information between rows and columns. Besides, the proposed pseudo label propagation algorithm can be a new method to construct anchor graphs with linear time complexity. A series of experiments are conducted on both synthetic and real-world datasets and the experimental results show the superior performance of three models. It is worth noting that for the proposed models, FMAWS2 is the generalization of FMAWS1 and FMAWS3 is the generalization of other two.

One-Step Multiview Clustering via Adaptive Graph Learning and Spectral Rotation.

In graph based multiview clustering methods, the ultimate partition result is usually achieved by spectral embedding of the consistent graph using some traditional clustering methods, such as k -means. However, optimal performance will be reduced by this multistep procedure since it cannot unify graph learning with partition generation closely. In this article, we propose a one-step multiview clustering method through adaptive graph learning and spectral rotation (AGLSR). For every view, AGLSR adaptively learns affinity graphs to capture similar relationships of samples. Then, a spectral embedding is designed to take advantage of the potential feature space shared by different views. In addition, AGLSR utilizes a spectral rotation strategy to obtain the discrete clustering labels from the learned spectral embeddings directly. An effective updating algorithm with proven convergence is derived to optimize the optimization problem. Sufficient experiments on benchmark datasets have clearly demonstrated the effectiveness of the proposed method in six metrics. The code of AGLSR is uploaded at https://github.com/tangchuan2000/AGLSR.

A Temporal Variable-Scale Clustering Method on Feature Identification for Policy Public-Opinion Management

The development of various digital social network platforms has caused public opinion to play an increasingly important role in the policy making process. However, due to the fact that public opinion hotspots usually change rapidly (such as the phenomenon of public opinion inversion), both the behaviour feature and demand feature of netizens included in the public opinion often vary over time. Therefore, this paper focuses on the feature identification problem of public opinion simultaneously considering the multiple observation time intervals and key time points, in order to support the management of policy-focused online public opinion. According to the variable-scale data analysis theory, the temporal scale space model is established to describe candidate temporal observation scales, which are organized following the time points of relevant policy promulgation (policy time points). After proposing the multi-scale temporal data model, a temporal variable-scale clustering method (T-VSC) is put forward. Compared to the traditional numerical variable-scale clustering method, the proposed T-VSC enables to combine the subjective attention of decision-makers and objective timeliness of public opinion data together during the scale transformation process. The case study collects 48552 raw public opinion data on the double-reduction education policy from Sina Weibo platform during Jan 2023 to Nov 2023. Experimental results indicate that the proposed T-VSC method could divide netizens that participate in the dissemination of policy-focused public opinion into clusters with low behavioural granularity deviation on the satisfied observation time scales, and identify the differentiated demand feature of each netizen cluster at policy time points, which could be applied to build the timely and efficient digital public dialogue mechanism.