Spectral Clustering Method Research Articles

A Multimodal Multi-Objective Coati Optimization Algorithm Based on Spectral Clustering

Multimodal multi-objective problems refer to situations where more than one Pareto set in the decision space corresponds to the same Pareto front in the objective space. In this paper, we propose a multimodal multi-objective coati optimization algorithm based on spectral clustering (MMOCOA-SC) for use in multimodal multi-objective problems. The algorithm starts by creating stable subpopulations via a spectral clustering method, grouping similar individuals within each subpopulation. Then, an improved coati optimization algorithm is proposed in order to increase the diversity of the population and continuously search for near-Pareto optimal solutions. Finally, we use an improved crowding distance method combined with a non-dominated sorting method to maintain and retain multiple near-Pareto optimal solutions. The MMOCOA-SC is evaluated alongside five state-of-the-art algorithms using the 2020 CEC test suite, IDMP test problems, and twelve classic engineering application problems. In comparison to the other five state-of-the-art algorithms, the experimental results show that MMOCOA-SC exhibits a superior performance when applied to the problems. This study explores multimodal multi-objective optimization algorithms with a focus on the concept of symmetry, which is crucial for multimodal multi-objective optimization problems in terms of achieving a balanced decision space and a diversity of solutions in the objective space.

A Novel Autoencoder-Integrated Clustering Methodology for Inventory Classification: A Real Case Study for White Goods Industry

This article presents an inventory classification method that provides more accurate results in the white goods factory, which will contribute to sustainability, sustainability economics, and supply chain management targets. A novel inventory classification application is presented with real-world data. Two different datasets are used, and these datasets are compared to each other. These larger dataset is Stock Keeping Unit (SKU)-based (6.032 SKUs), and the smaller one is product-group-based (270 product groups). In the first phase, Artificial Intelligence (AI) clustering methods that have not been used in the field of inventory classification, to our knowledge, are applied to these datasets; the results are obtained and compared using K-Means, Gaussian mixture, agglomerative clustering, and spectral clustering methods. In the second stage, an autoencoder is separately hybridized with the AI clustering methods to develop a novel approach to inventory classification. Fuzzy C-Means (FCM) is used in the third step to classify inventories. At the end of the study, these nine different methodologies (“K-Means, Gaussian mixture, agglomerative clustering, spectral clustering” with and without the autoencoder and Fuzzy C-Means) are compared using two different datasets. It is shown that the proposed new hybrid method gives much better results than classical AI methods.

Optimized Spectral Clustering Methods For Potentially Divergent Biological Sequences

Various recent researches in bioinformatics demonstrated that clustering is a very efficient technique for sequence analysis. Spectral clustering is particularly efficient for highly divergent sequences and GMMs (Gaussian Mixture Models) are often able to cluster overlapping groups if given an adequately designed embedding. The current study used spectral embedding and Mixture Models for clustering potentially divergent biological sequences. The research approach resulted in a pipeline consisting of the following four steps. The first step consists of aligning the biological sequences. The pairwise affinity of the sequences is computed in the second step. Then the Laplacian Eigenmap embedding of the data is performed in the third step. Finally, the last step consists of a GMM-based clustering. Improving the quality of the generated clustering and the performance of this approach is directly related to the enhancement of each one of these four steps. The main contribution is proposing four GMM-based algorithms for automatically selecting the optimal number of clusters and optimizing the clustering quality. A clustering quality assessment method, based on phylogenetic trees, is also proposed. Moreover, a performance study and analysis have been conducted while testing different clustering methods and GMM implementations. Experimental results demonstrated the superiority of using the BIC (Bayesian Information Criterion) for selecting the optimal GMM configuration. Significant processing speed improvements were also recorded for the implementation of the proposed algorithms.

Auto-weighted Graph Reconstruction for efficient ensemble clustering

Ensemble clustering (EC) is an important field in data clustering, which aims to combine multiple base clustering results of a given dataset into a single consensus result. Previous studies have shown that one of the effective methods is to convert the base clustering results into a graph partitioning problem. Existing spectral ensemble clustering methods typically obtain a spectral embedding matrix for the graph and then perform k-means, separating the graph construction and clustering tasks into two distinct stages. To address these challenges, we propose a novel and efficient ensemble clustering method named Auto-weighted Graph Reconstruction for efficient ensemble clustering. This method integrates weighted base clusters, consensus distance graph construction, and clustering into a unified framework. A consensus distance graph is constructed, from which a stable and discrete label matrix is derived using spectral clustering. An effective optimization algorithm is then employed to address the resulting problem. Finally, the experimental results on eight real-world datasets verify the effectiveness and superiority of the proposed method.

Clustering-based hyper-heuristic algorithm for multi-region coverage path planning of heterogeneous UAVs

In the context of multi-heterogeneous UAV coverage path planning, an effective solution method has been proposed. Firstly, regions are set up as fully connected graphs which are cut into multiple subgraphs by spectral clustering method to assign tasks to multi-heterogeneous UAVs. Additionally, an RL-based hyper-heuristic algorithm is proposed. Heuristic space is parameterized by GNN which is trained with the reward provided by the optimization goal to automate design and enhance the heuristic metrics, avoiding the inefficiency and suboptimality of expert design and manual parameter tuning. Compared with existing methods, the proposed algorithm has a better performance in task completion time, execution time and deviation rate, which shows its potential application in the coverage path planning problem of multi-heterogeneous UAVs.

Unsupervised selective labeling for semi-supervised industrial defect detection

In industrial detection scenarios, achieving high accuracy typically relies on extensive labeled datasets, which are costly and time-consuming. This has motivated a shift towards semi-supervised learning (SSL), which leverages labeled and unlabeled data to improve learning efficiency and reduce annotation costs. This work proposes the unsupervised spectral clustering labeling (USCL) method to optimize SSL for industrial challenges like defect variability, rarity, and complex distributions. Integral to USCL, we employ the multi-task fusion self-supervised learning (MTSL) method to extract robust feature representations through multiple self-supervised tasks. Additionally, we introduce the Enhanced Spectral Clustering (ESC) method and a dynamic selecting function (DSF). ESC effectively integrates both local and global similarity matrices, improving clustering accuracy. The DSF maximally selects the most valuable instances for labeling, significantly enhancing the representativeness and diversity of the labeled data. USCL consistently improves various SSL methods compared to traditional instance selection methods. For example, it boosts Efficient Teacher by 5%, 6.6%, and 7.8% in mean Average Precision(mAP) on the Automotive Sealing Rings Defect Dataset, the Metallic Surface Defect Dataset, and the Printed Circuit Boards (PCB) Defect Dataset with 10% labeled data. Our work sets a new benchmark for SSL in industrial settings.

An improved spectral clustering method for accurate detection of brain resting-state networks

This paper proposes a data-driven analysis method to accurately partition large-scale resting-state functional brain networks from fMRI data. The method is based on a spectral clustering algorithm and combines eigenvector direction selection with Pearson correlation clustering in the spectral space. The method is an improvement on available spectral clustering methods, capable of robustly identifying active brain networks consistent with those from model-driven methods at different noise levels, even at the noise level of real fMRI data.

Greedy recursive spectral bisection for modularity-bound hierarchical divisive community detection

Spectral clustering techniques depend on the eigenstructure of a similarity matrix to assign data points to clusters, so that points within the same cluster exhibit high similarity and are compared to those in different clusters. This work aimed to develop a spectral method that could be compared to clustering algorithms that represent the current state of the art. This investigation conceived a novel spectral clustering method, as well as five policies that guide its execution, based on spectral graph theory and embodying hierarchical clustering principles. Computational experiments comparing the proposed method with six state-of-the-art algorithms were undertaken in this study to evaluate the clustering methods under scrutiny. The assessment was performed using two evaluation metrics, specifically the adjusted Rand index, and modularity. The obtained results furnish compelling evidence, indicating that the proposed method is competitive and possesses distinctive properties compared to those elucidated in the existing literature. This suggests that our approach stands as a viable alternative, offering a robust choice within the spectrum of available same-purpose tools.

Robust Tensor Learning for Multi-View Spectral Clustering

Tensor-based multi-view spectral clustering methods are promising in practical clustering applications. However, most of the existing methods adopt the ℓ2,1 norm to depict the sparsity of the error matrix, and they usually ignore the global structure embedded in each single view, compromising the clustering performance. Here, we design a robust tensor learning method for multi-view spectral clustering (RTL-MSC), which employs the weighted tensor nuclear norm to regularize the essential tensor for exploiting the high-order correlations underlying multiple views and adopts the nuclear norm to constrain each frontal slice of the essential tensor as the block diagonal matrix. Simultaneously, a novel column-wise sparse norm, namely, ℓ2,p, is defined in RTL-MSC to measure the error tensor, making it sparser than the one derived by the ℓ2,1 norm. We design an effective optimization algorithm to solve the proposed model. Experiments on three widely used datasets demonstrate the superiority of our method.

Classification of high-dimensional imbalanced biomedical data based on spectral clustering SMOTE and marine predators algorithm

The research of biomedical data is crucial for disease diagnosis, health management, and medicine development. However, biomedical data are usually characterized by high dimensionality and class imbalance, which increase computational cost and affect the classification performance of minority class, making accurate classification difficult. In this paper, we propose a biomedical data classification method based on feature selection and data resampling. First, use the minimal-redundancy maximal-relevance (mRMR) method to select biomedical data features, reduce the feature dimension, reduce the computational cost, and improve the generalization ability; then, a new SMOTE oversampling method (Spectral-SMOTE) is proposed, which solves the noise sensitivity problem of SMOTE by an improved spectral clustering method; finally, the marine predators algorithm is improved using piecewise linear chaotic maps and random opposition-based learning strategy to improve the algorithm’s optimization seeking ability and convergence speed, and the key parameters of the spectral-SMOTE are optimized using the improved marine predators algorithm, which effectively improves the performance of the over-sampling approach. In this paper, five real biomedical datasets are selected to test and evaluate the proposed method using four classifiers, and three evaluation metrics are used to compare with seven data resampling methods. The experimental results show that the method effectively improves the classification performance of biomedical data. Statistical test results also show that the proposed PRMPA-Spectral-SMOTE method outperforms other data resampling methods.

A Comprehensive Survey on Community Detection With Deep Learning.

Detecting a community in a network is a matter of discerning the distinct features and connections of a group of members that are different from those in other communities. The ability to do this is of great significance in network analysis. However, beyond the classic spectral clustering and statistical inference methods, there have been significant developments with deep learning techniques for community detection in recent years--particularly when it comes to handling high-dimensional network data. Hence, a comprehensive review of the latest progress in community detection through deep learning is timely. To frame the survey, we have devised a new taxonomy covering different state-of-the-art methods, including deep learning models based on deep neural networks (DNNs), deep nonnegative matrix factorization, and deep sparse filtering. The main category, i.e., DNNs, is further divided into convolutional networks, graph attention networks, generative adversarial networks, and autoencoders. The popular benchmark datasets, evaluation metrics, and open-source implementations to address experimentation settings are also summarized. This is followed by a discussion on the practical applications of community detection in various domains. The survey concludes with suggestions of challenging topics that would make for fruitful future research directions in this fast-growing deep learning field.

Enhancing generalized spectral clustering with embedding Laplacian graph regularization

AbstractAn enhanced generalised spectral clustering framework that addresses the limitations of existing methods by incorporating the Laplacian graph and group effect into a regularisation term is presented. By doing so, the framework significantly enhances discrimination power and proves highly effective in handling noisy data. Its versatility enables its application to various clustering problems, making it a valuable contribution to unsupervised learning tasks. To optimise the proposed model, the authors have developed an efficient algorithm that utilises the standard Sylvester equation to compute the coefficient matrix. By setting the derivatives to zero, computational efficiency is maintained without compromising accuracy. Additionally, the authors have introduced smoothing strategies to make the non‐convex and non‐smooth terms differentiable. This enables the use of an alternative iteration re‐weighted procedure (AIwRP), which distinguishes itself from other first‐order optimisation algorithms by introducing auxiliary variables. The authors provide a provable convergence analysis of AIwRP based on the iteration procedures of unconstrained problems to support its effectiveness. Extensive numerical tests have been conducted on synthetic and benchmark databases to validate the superiority of their approaches. The results demonstrate improved clustering performance and computational efficiency compared to several existing spectral clustering methods, further reinforcing the advantages of their proposed framework. The source code is available at https://github.com/ZhangHengMin/LGR_LSRLRR.

Eigenvalue-Based Incremental Spectral Clustering

Abstract Our previous experiments demonstrated that subsets of collections of (short) documents (with several hundred entries) share a common, normalized in some way, eigenvalue spectrum of combinatorial Laplacian. Based on this insight, we propose a method of incremental spectral clustering. The method consists of the following steps: (1) split the data into manageable subsets, (2) cluster each of the subsets, (3) merge clusters from different subsets based on the eigenvalue spectrum similarity to form clusters of the entire set. This method can be especially useful for clustering methods of complexity strongly increasing with the size of the data sample, like in case of typical spectral clustering. Experiments were performed showing that in fact the clustering and merging of subsets yield clusters close to clustering of the entire dataset. Our approach differs from other research streams in that we rely on the entire set (spectrum) of eigenvalues, whereas the other researchers concentrate on few eigenvectors related to lowest eigenvalues. Such eigenvectors are considered in the literature as of low reliability.

Seasonal types in homogeneous rainfall regions of the Amazon basin

AbstractDue to its size and geographical features, different average annual rainfall regimes co‐exist in the Amazon basin, with distinct year‐to‐year variability dependent on regions within the basin. In this study, we define and explain the seasonal regional types of annual regimes, that is, years with similar seasonal anomalies. Our work is based on a 205 rain gauge network distributed over five Amazonian countries, spanning a period over 30 years. Using a spectral clustering method, we identified seven sub‐regions within the basin in which annual rainfall regimes are spatially homogenous. For each sub‐domain, we estimated specific parameters that characterize the rainy season (onset and demise dates, sign and duration of rainfall anomalies). Finally, using spectral analysis we identified between two and four ‘seasonal type’ of precipitation in these seven sub‐domains. Most of these seasonal types are in phase with the large‐scale atmospheric circulation, which explains the temporal link with rainfall anomalies. The seasonal types result of the superposition of inter‐annual and intra‐seasonal variability whose factors are then difficult to identify and attribute. Part of the rainfall anomalies characterizing seasonal types is related to the inter‐annual variability of the sea surface temperature in the Atlantic or the Pacific oceans, especially in the northeast and southeast part of the Amazon basin, whereas in other parts, strong intra‐seasonal and local factors have a larger impact. The same sign and duration of anomalies do not concomitantly affect the various regions of the Amazon basin, confirming that one mode of variability does not homogeneously affect precipitation in different parts of the basin.

Recognizing and detecting COVID-19 in chest X-ray images using constrained multi-view spectral clustering

AbstractMachine learning, particularly classification algorithms, has been widely employed for diagnosing COVID-19 cases. However, these methods typically rely on labeled datasets and analyze a single data view. With the vast amount of patient data available without labels, this paper addresses the novel challenge of unsupervised COVID-19 diagnosis. The goal is to harness the abundant data without labels effectively. In recent times, multi-view clustering has garnered considerable attention in the research community. Spectral clustering, known for its robust theoretical framework, is a key focus. However, traditional spectral clustering methods generate only nonlinear data projections, necessitating additional clustering steps. The quality of these post-processing steps can be influenced by various factors, such as initialization procedures and outliers. This paper introduces an enhanced version of the recent “Multiview Spectral Clustering via integrating Nonnegative Embedding and Spectral Embedding” method. While retaining the benefits of the original technique, the proposed model integrates two essential constraints: (1) a constraint for ensuring the consistent smoothness of the nonnegative embedding across all views and (2) an orthogonality constraint imposed on the columns of the nonnegative embedding matrix. The effectiveness of this approach is demonstrated using COVIDx datasets. Additionally, the method is evaluated on other image datasets to validate its suitability for this study.

A Similarity Matrix Low-Rank Approximation and Inconsistency Separation Fusion Approach for Multiview Clustering

In recent years, multi-view clustering algorithms have achieved promising performance by exploiting the com-plementarity and consistency of different views. However, many multi-view spectral clustering methods only focus on the con-sistent information of views, and the time cost of feature decomposition is expensive. Moreover, these methods also require post-processing (e.g., k-means) to obtain the final clustering results. To overcome these limitations simultaneously, we propose a novel multi-view clustering algorithm. Firstly, the method removes the inconsistent information of the views through cross-view measurement to maintain consistent information. These inconsistencies may be caused by noise, corruptions, or view-specific properties and will affect the quality of the similarity matrix. Then, we learn a consensus embedding matrix with non-negative constraints by performing a low-rank decomposition of the consistency information. In this way, we can replace the eigendecomposition of the n × n Laplacian matrix in spectral clustering with the singular value decomposition of a n × c low-rank matrix to reduce the computational burden, where c ≪ n. Furthermore, due to the non-negative constraint, we can directly obtain the clustering results. Also, to consider the diversity of views, adaptive weighting is applied to different view data. Compared to state-of-the-art multi-view clustering methods on five benchmark multi-view datasets, we demonstrate the superiority and effectiveness of our approach. We release the source code at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/hulu88/FAMvC</uri> .

Data-Driven Outage Restoration Time Prediction via Transfer Learning With Cluster Ensembles

This paper develops a data-driven approach to accurately predict the restoration time of outages under different scales and factors. To achieve the goal, the proposed method consists of three stages. First, given the unprecedented amount of data collected by utilities, a sparse dictionary-based ensemble spectral clustering (SDESC) method is proposed to decompose historical outage datasets, which enjoys good computational efficiency and scalability. Specifically, each outage sample is represented by a linear combination of a small number of selected dictionary samples using a density-based method. Then, the dictionary-based representation is utilized to perform the spectral analysis to group the data samples with similar features into the same subsets. In the second stage, a knowledge-transfer-added restoration time prediction model is trained for each subset by combining weather information and outage-related features. The transfer learning technology is introduced to deal with the underestimation problem caused by data imbalance in different subsets, thus improving the model performance. Furthermore, to connect unseen outages with the learned outage subsets, a t-distributed stochastic neighbor embedding-based strategy is applied. The proposed method fully builds on and is also tested on a large real-world outage dataset from a utility provider with a time span of six consecutive years. The numerical results validate that our method has high prediction accuracy while showing good stability against real-world data limitations.

A Novel and Effective Method to Directly Solve Spectral Clustering.

Spectral clustering has been attracting increasing attention due to its well-defined framework and excellent performance. However, most traditional spectral clustering methods consist of two separate steps: 1) Solving a relaxed optimization problem to learn the continuous clustering labels, and 2) Rounding the continuous clustering labels into discrete ones. The clustering results of the relax-and-discretize strategy inevitably result in information loss and unsatisfactory clustering performance. Moreover, the similarity matrix constructed from original data may not be optimal for clustering since data usually have noise and redundancy. To address these problems, we propose a novel and effective algorithm to directly optimize the original spectral clustering model, called Direct Spectral Clustering (DSC). We theoretically prove that the original spectral clustering model can be solved by simultaneously learning a weighted discrete indicator matrix and a structured similarity matrix whose connected components are equal to the number of clusters. Both of them can be used to directly obtain the final clustering results without any post-processing. Further, an effective iterative optimization algorithm is exploited to solve the proposed method. Extensive experiments performed on synthetic and real-world datasets demonstrate the superiority and effectiveness of the proposed method compared to the state-of-the-art algorithms.

Modelling and application of a spectral clustering method for shared bicycle trajectories

Geographic flow clustering analysis can effectively reveal human behavioral patterns in movement. Traditional methods for studying human movement patterns are mostly based on first-order quantity analyses of point data, such as hotspots, density or clustering. Currently, relatively few second-order spatial analysis methods based on geographic flows exist. Thus, we developed a new geographic flow method based on spectral clustering and applied it to trajectory data analysis. This article uses the bike-sharing trajectories data in Shanghai in August 2016, spectral clustering analysis was conducted on the group flow patterns before, during and after rainfall, on weekdays and weekends and in the morning and evening peak. Spectral clustering was verified to exhibit better clustering effect by comparing the clustering indices of different clustering methods. This study enriches the analysis method of geographical flows, and the human mobility patterns revealed by its analysis can provide references for formulating urban green travel policies.

Towards explainability of hashtags in the light of Graph Spectral Clustering methods

Towards explainability of hashtags in the light of Graph Spectral Clustering methods