K-nearest Neighbor Graph Research Articles

GSSCL: A framework for Graph Self-Supervised Curriculum Learning based on clustering label smoothing

Graph self-supervised learning is an effective technique for learning common knowledge from unlabeled graph data through pretext tasks. To capture the interrelationships between nodes and their essential roles globally, existing methods use clustering labels as self-supervised signals. However, in some cases, these methods may introduce noise, leading to over-fitting of the model and a reduction in performance. To address these issues, a novel framework for Graph Self-Supervised Curriculum Learning based on clustering label smoothing called GSSCL has been proposed. GSSCL clusters knowledge in an easy-to-difficult manner, reducing the heavy dependence on the reliability of clustering and improving the generalizability of the model. Moreover, the Silhouette Coefficient is employed to evaluate the clustering confident scores for all nodes. Some nodes are selected based on high confident scores to perform self-supervised learning. To account for the possibility of complex heterophilous information in graphs (e.g., noisy links), clustering pseudo-label smoothing is performed on K-nearest neighbor graphs built upon the similarities between node features instead of the original graph structures. The obtained multi-scale knowledge is then applied to curriculum learning. Finally, comprehensive experiments conducted across diverse public graph benchmarks demonstrate the superior performance of the proposed framework. It exhibits comparable results to state-of-the-art methods across semi-supervised node classification and clustering tasks.

Pair-EGRET: Enhancing the prediction of protein-protein interaction sites through graph attention networks and protein language models.

Proteins are responsible for most biological functions, many of which require the interaction of more than one protein molecule. However, accurately predicting protein-protein interaction (PPI) sites (the interfacial residues of a protein that interact with other protein molecules) remains a challenge. The growing demand and cost associated with the reliable identification of PPI sites using conventional experimental methods call for computational tools for automated prediction and understanding of PPIs. We present Pair-EGRET, an edge-aggregated graph attention network that leverages the features extracted from pre-trained transformer-like models to accurately predict PPI sites. Pair-EGRET works on a k-nearest neighbor graph, representing the three-dimensional structure of a protein, and utilizes the cross-attention mechanism for accurate identification of interfacial residues of a pair of proteins. Through an extensive evaluation study using a diverse array of experimental data, evaluation metrics, and case studies on representative protein sequences, we demonstrate that Pair-EGRET can achieve remarkable performance in predicting PPI sites. Moreover, Pair-EGRET can provide interpretable insights from the learned cross-attention matrix. Pair-EGRET is freely available in open source form at the GitHub Repository https://github.com/1705004/Pair-EGRET. Supplementary data are available at Bioinformatics online.

A simulation data-driven semi-supervised framework based on MK-KNN graph and ESSGAT for bearing fault diagnosis

Current supervised intelligent fault diagnosis relies on abundant labeled data. However, collecting and labeling data are typically both expensive and time-consuming. Fault diagnosis with unlabeled data remains a significant challenge. To address this issue, a simulation data-driven semi-supervised framework based on multi-kernel K-nearest neighbor (MK-KNN) and edge self-supervised graph attention network (ESSGAT) is proposed. The novel MK-KNN establishes the neighborhood relationships between simulation data and real data. The developed multi-kernel function mitigates the risks of overfitting and underfitting, thereby enhancing the robustness of the simulation-real graphs. The designed ESSGAT employs two forms of self-supervised attention to predict the presence of edges, increasing the weights of crucial neighboring nodes in the MK-KNN graph. The performance of the proposed method is evaluated using a public bearing dataset and a self-constructed dataset of high-speed train axle box bearings. The results show that the proposed method achieves better diagnostic performance compared with other state-of-the-art graph construction methods and graph convolutional networks.

Community detection using Jaya optimization algorithm based on deep learning methods and KNN graph-based clustering

Purpose In this paper, community identification has been considered as the most critical task of social network analysis. The purpose of this paper is to organize the nodes of a given network graph into distinct clusters or known communities. These clusters will therefore form the different communities available within the social network graph. Design/methodology/approach To date, numerous methods have been developed to detect communities in social networks through graph clustering techniques. The k-means algorithm stands out as one of the most well-known graph clustering algorithms, celebrated for its straightforward implementation and rapid processing. However, it has a serious drawback because it is insensitive to initial conditions and always settles on local optima rather than finding the global optimum. More recently, clustering algorithms that use a reciprocal KNN (k-nearest neighbors) graph have been used for data clustering. It skillfully overcomes many major shortcomings of k-means algorithms, especially about the selection of the initial centers of clusters. However, it does face its own challenge: sensitivity to the choice of the neighborhood size parameter k, which is crucial for selecting the nearest neighbors during the clustering process. In this design, the Jaya optimization method is used to select the K parameter in the KNN method. Findings The experiment on real-world network data results show that the proposed approach significantly improves the accuracy of methods in community detection in social networks. On the other hand, it seems to offer some potential for discovering a more refined hierarchy in social networks and thus becomes a useful tool in the analysis of social networks. Originality/value This paper introduces an enhancement to the KNN graph-based clustering method by proposing a local average vector method for selecting the optimal neighborhood size parameter k. Furthermore, it presents an improved Jaya algorithm with KNN graph-based clustering for more effective community detection in social network graphs.

Asymmetry index for data and its verification in dimensionality reduction and data visualization

We propose an asymmetry index as a measure of degree of asymmetry of a given dataset. It provides an additional information on a dataset allowing to guide and improve any further analysis. The index reflects the intensity of the asymmetric relationships among data resulting from hierarchical data structure. Using the information retrieved by our asymmetry index, one obtains a justification and explanation of the effectiveness of the subsequent asymmetric data analysis methods, as well as helpful preparation to asymmetrizing the tools for the further analysis. The asymmetry index is based on the k-nearest neighbors graph representing the considered data. Therefore, it uses the intrinsic geometry-based information on the data, in this way, providing an insight into the data structure. Our experiments on real data are designed to verify the usefulness of the asymmetry index and the correctness of its theoretical fundamentals. In our empirical validation, we employ the symmetric and asymmetric dimensionality reduction algorithms and evaluate their results on the basis of clustering in the 2-dimensional visualization space. We test, whether our index indeed predicts the level of superiority of the asymmetric methods over their symmetric counterparts.

AKNNO: single-cell and spatial transcriptomics clustering with an optimized adaptive k-nearest neighbor graph

Typical clustering methods for single-cell and spatial transcriptomics struggle to identify rare cell types, while approaches tailored to detect rare cell types gain this ability at the cost of poorer performance for grouping abundant ones. Here, we develop aKNNO to simultaneously identify abundant and rare cell types based on an adaptive k-nearest neighbor graph with optimization. Benchmarking on 38 simulated and 20 single-cell and spatial transcriptomics datasets demonstrates that aKNNO identifies both abundant and rare cell types more accurately than general and specialized methods. Using only gene expression aKNNO maps abundant and rare cells more precisely compared to integrative approaches.

Diffusion on PCA-UMAP Manifold: The Impact of Data Structure Preservation to Denoise High-Dimensional Single-Cell RNA Sequencing Data.

Single-cell transcriptomics (scRNA-seq) is revolutionizing biological research, yet it faces challenges such as inefficient transcript capture and noise. To address these challenges, methods like neighbor averaging or graph diffusion are used. These methods often rely on k-nearest neighbor graphs from low-dimensional manifolds. However, scRNA-seq data suffer from the 'curse of dimensionality', leading to the over-smoothing of data when using imputation methods. To overcome this, sc-PHENIX employs a PCA-UMAP diffusion method, which enhances the preservation of data structures and allows for a refined use of PCA dimensions and diffusion parameters (e.g., k-nearest neighbors, exponentiation of the Markov matrix) to minimize noise introduction. This approach enables a more accurate construction of the exponentiated Markov matrix (cell neighborhood graph), surpassing methods like MAGIC. sc-PHENIX significantly mitigates over-smoothing, as validated through various scRNA-seq datasets, demonstrating improved cell phenotype representation. Applied to a multicellular tumor spheroid dataset, sc-PHENIX identified known extreme phenotype states, showcasing its effectiveness. sc-PHENIX is open-source and available for use and modification.

Few-shot bearing fault diagnosis by semi-supervised meta-learning with graph convolutional neural network under variable working conditions

Aiming at the problems of low accuracy and weak generalization ability in fault diagnosis caused by complex working conditions and limited fault samples of bearings, a few-shot bearing fault diagnosis method by semi-supervised meta-learning with simplifying graph convolutional neural network (semi-meta-sgc) is proposed. Firstly, the time domain signal is converted into the frequency domain by fast Fourier transform (FFT). Secondly, considering each spectrum sample as a node, a k-nearest neighbor (KNN) graph is constructed according to the adjacencies between nodes, thus transforming the bearing fault classification into a graphical node classification. Then, relying on meta-learning, graph convolutional neural network, and semi-supervised learning, a high-accuracy node classifier is obtained by using a small number of training samples to achieve the classification of bearing faults under variable working conditions. Finally, the proposed method is verified by four cases and compared with other methods. The results show that this method has higher recognition accuracy and generalization performance.

Density peaks clustering based on Gaussian fuzzy neighborhood with noise parameter

Density peak clustering (DPC) is an effective clustering method known for its robustness, non-iterative nature, and hybrid approach. However, it is not without limitations: (a) the determination of the cutoff distance (dc) relies on human experience, which can significantly impact the clustering outcome; (b) DPC does not take into account the local structure of the data when computing local densities; (c) it employs a crisp kernel for density computation; (d) the performance of DPC is affected by chain reactions; and (e) DPC often struggles to handle noisy data. In order to address these limitations, this paper proposes a novel approach called DPC based on Gaussian fuzzy neighborhood with noise parameter (DPC-GFNN). The proposed method leverages a Gaussian fuzzy kernel to enhance the separation between clusters and mitigates the influence of outliers through an adjustable noise parameter (λ). DPC-GFNN utilizes a k-nearest neighbor graph based on density to label highly dense regions. This technique effectively avoids chain reactions by assigning accurate labels to points in border areas, enabling proper clustering of data with diverse shapes and densities. To evaluate the effectiveness of DPC-GFNN, a series of experiments are conducted on both real-world and synthetic datasets. The experimental results demonstrate that DPC-GFNN exhibits superior robustness and clustering accuracy compared to other modified variants of DPC including DPC based on k-nearest neighbors (DPC-KNN), improved DPC (IDPC), DPC based on density backbone and fuzzy neighborhood (DPC-DBFN), and DPC based on fuzzy weighted k-nearest neighbors (FKNN).

Mechanical Fault Diagnosis of High-Voltage Circuit Breakers with Dynamic Multi-Attention Graph Convolutional Networks Based on Adaptive Graph Construction

With the rapid development of deep learning, its powerful capabilities make it possible to perform mechanical fault diagnosis of high-voltage circuit breakers (HVCBs). Among deep learning approaches, the convolutional neural network is widely used. However, while it can extract features effectively, it also has some limitations. Specifically, it depends on a large number of training data and only takes data information into account without considering structural information. These shortcomings lead to unused information and unsatisfactory model results. To address these shortcomings, this paper proposes AKNN-DMGCN, a novel dynamic multi-attention graph convolutional network based on an adaptively constructed graph, which can achieve high accuracy and robust mechanical fault diagnosis of HVCBs. First, a novel adaptive k-nearest neighbor (AKNN) graph construction method is proposed to construct informative graphs. The AKNN method can mine the relationship between the original data samples and utilize the data and label information. Thus, it has high fault tolerance to noise signals and can construct a structure graph with rich and accurate information, which can improve the overall model performance. Then, a dynamic multi-attention graph convolutional network (DMGCN) is applied for mechanical fault diagnosis of HVCBs. DMGCN fully utilizes structural and numerical information representing HVCB signals to perform classification. DMGCN has a dynamic multi-attention mechanism with strong expressive ability, which allows it to achieve high diagnostic accuracy. The experimental results indicate that the accuracy of AKNN-DMGCN reaches 97.22% on a balanced dataset and 95.01% on an imbalanced dataset, which demonstrates that the proposed method is effective for both balanced and imbalanced samples.

Semi-supervised few-shot fault diagnosis driven by multi-head dynamic graph attention network under speed fluctuations

Although graph neural networks (GNNs) have achieved great success in the field of semi-supervised bearing fault diagnosis recently, most GNN-based studies still face the challenge of insufficient labeled samples. Meanwhile, the signals collected from bearings usually have a time offset due to the alternating speed in the actual working conditions, reducing the applicability of traditional GNN diagnosis methods relying on Euclidean distance. And incomplete use of unlabeled data and insufficient exploitation of neighborhood information further reduce the reliability of fault diagnosis models. To solve these problems, a semi-supervised few-shot fault diagnosis method driven by multi-head dynamic graph attention network (MHDGAT) is proposed in this study. First, dynamic time warping (DTW) similarity is introduced in the construction of K-nearest neighbor graph to solve the problem of local time offset caused by speed change. Second, some pseudo labels generated by smoothness assumption are injected into the raw training set, and more comprehensive information can be learned in the process of model training. Then, the presented MHDGAT model can mine rich fault characteristics from different scales by the multi-head attention and extract the most sensitive features due to the proposed dynamic graph mechanism adaptively. Finally, the fault categories can be obtained by a softmax classifier located at the end of the model. The proposed model has been validated in two case studies, and the experimental results indicate that MHDGAT exerts strong performance even under extremely low-label ratios.

Heterogeneous graph community detection method based on K-nearest neighbor graph neural network

Traditional community detection models either ignore the feature space information and require a large amount of domain knowledge to define the meta-paths manually, or fail to distinguish the importance of different meta-paths. To overcome these limitations, we propose a novel heterogeneous graph community detection method (called KGNN_HCD, heterogeneous graph Community Detection method based on K-nearest neighbor Graph Neural Network). Firstly, the similarity matrix is generated to construct the topological structure of K-nearest neighbor graph; secondly, the meta-path information matrix is generated using a meta-path transformation layer (Mp-Trans Layer) by adding weighted convolution; finally, a graph convolutional network (GCN) is used to learn high-quality node representation, and the k-means algorithm is adopted on node embeddings to detect the community structure. We perform extensive experiments and on three heterogeneous datasets, ACM, DBLP and IMDB, and we consider as competitors 11 community detection methods such as CP-GNN and GTN. The experimental results show that the proposed KGNN_HCD method improves 2.54% and 2.56% on the ACM dataset, 2.59% and 1.47% on the DBLP dataset, and 1.22% and 1.67% on the IMDB dataset for both NMI and ARI. Experiments findings suggest that the proposed KGNN_HCD method is reasonable and effective, and KGNN_HCD can be applied to complex network classification and clustering tasks.

Online course evaluation model based on graph auto-encoder

In the post-epidemic era, online learning has gained increasing attention due to the advancements in information and big data technology, leading to large-scale online course data with various student behaviors. Online data mining has become a popular and important way of extracting valuable insights from large amounts of data. However, previous online course analysis methods often focused on individual aspects of the data and neglected the correlation among the large-scale learning behavior data, which can lead to an incomplete understanding of the overall learning behavior and patterns within the online course. To solve the problems, this paper proposes an online course evaluation model based on a graph auto-encoder. In our method, the features of collected online course data are used to construct K-Nearest Neighbor(KNN) graphs to represent the association among the courses. Then the variational graph auto-encoder(VGAE) is introduced to learn the useful implicit features. Finally, we feed the learned implicit features into unsupervised and semi-supervised downstream tasks for online course evaluation, respectively. We conduct experiments on two datasets. In the clustering task, our method showed a more than tenfold increase in the Calinski-Harabasz index compared to unoptimized features, demonstrating significant structural distinction and group coherence. In the classification task, compared to traditional methods, our model exhibited an overall performance improvement of about 10%, indicating its effectiveness in handling complex network data.

Intelligent fault diagnosis of ultrasonic motors based on graph-regularized CNN-BiLSTM

The ultrasonic motor (USM) is peculiarly prone to failure due to continuous high-frequency friction-related power transfer, whose failure mechanisms are remarkably different from traditional induction motors. Intelligent fault diagnosis provides a way to alarm and avoid catastrophic losses proactively. However, previous studies using deep learning usually ignore the inherent geometric structure of the signal distribution. This paper proposes an intelligent multi-signal fault diagnosis framework for USMs to restore the linear or nonlinear manifold structure by preserving the internal structure by integrating graph regularization with deep neural networks. Firstly, the one-dimensional CNN to learn spatial correlations and BiLSTM to exploit temporal dependencies are coalesced to build the deep neural network. Then, an improved k-nearest neighbor graph is proposed to protect the geometric structure information and force the latent features to be more concentrated within their classes. Moreover, the layer in the deep architecture to integrate graph regularization is designed to reduce computation cost, and an adaptive decay strategy is considered to adjust the coefficient of graph regularized automatically. A two-stage training algorithm is developed by considering the time to calculate the graph regularization term. Finally, the proposed multi-signal fault diagnosis framework is validated using datasets from the fault injection experiment of similar USMs in China’s Yutu rover of Chang’e lunar probe. Experimental results show that the proposed method can effectively discriminate different fault types.

CDSKNNXMBD: a novel clustering framework for large-scale single-cell data based on a stable graph structure

BackgroundAccurate and efficient cell grouping is essential for analyzing single-cell transcriptome sequencing (scRNA-seq) data. However, the existing clustering techniques often struggle to provide timely and accurate cell type groupings when dealing with datasets with large-scale or imbalanced cell types. Therefore, there is a need for improved methods that can handle the increasing size of scRNA-seq datasets while maintaining high accuracy and efficiency.MethodsWe propose CDSKNNXMBD (Community Detection based on a Stable K-Nearest Neighbor Graph Structure), a novel single-cell clustering framework integrating partition clustering algorithm and community detection algorithm, which achieves accurate and fast cell type grouping by finding a stable graph structure.ResultsWe evaluated the effectiveness of our approach by analyzing 15 tissues from the human fetal atlas. Compared to existing methods, CDSKNN effectively counteracts the high imbalance in single-cell data, enabling effective clustering. Furthermore, we conducted comparisons across multiple single-cell datasets from different studies and sequencing techniques. CDSKNN is of high applicability and robustness, and capable of balancing the complexities of across diverse types of data. Most importantly, CDSKNN exhibits higher operational efficiency on datasets at the million-cell scale, requiring an average of only 6.33 min for clustering 1.46 million single cells, saving 33.3% to 99% of running time compared to those of existing methods.ConclusionsThe CDSKNN is a flexible, resilient, and promising clustering tool that is particularly suitable for clustering imbalanced data and demonstrates high efficiency on large-scale scRNA-seq datasets.

A Multi-Scale Graph Based on Spatio-Temporal-Radiometric Interaction for SAR Image Change Detection

Change detection (CD) in remote sensing imagery has found broad applications in ecosystem service assessment, disaster evaluation, urban planning, land utilization, etc. In this paper, we propose a novel graph model-based method for synthetic aperture radar (SAR) image CD. To mitigate the influence of speckle noise on SAR image CD, we opt for comparing the structures of multi-temporal images instead of the conventional approach of directly comparing pixel values, which is more robust to the speckle noise. Specifically, we first segment the multi-temporal images into square patches at multiple scales and construct multi-scale K-nearest neighbor (KNN) graphs for each image, and then develop an effective graph fusion strategy, facilitating the exploitation of multi-scale information within SAR images, which offers an enhanced representation of the complex relationships among features in the images. Second, we accomplish the interaction of spatio-temporal-radiometric information between graph models through graph mapping, which can efficiently uncover the connections between multi-temporal images, leading to a more precise extraction of changes between the images. Finally, we use the Markov random field (MRF) based segmentation method to obtain the binary change map. Through extensive experimentation on real datasets, we demonstrate the remarkable superiority of our methodologies by comparing with some current state-of-the-art methods.

A novel extended rule-based system based on K-Nearest Neighbor graph

The Belief Rule-Based (BRB) system faces the rule combination explosion issue, making it challenging to construct the rule base efficiently. The Extended Belief Rule-Based (EBRB) system offers a solution to this problem by using data-driven methods. However, using EBRB system requires the traversal of the entire rule base, which can be time-consuming and result in the activation of many irrelevant rules, leading to an incorrect decision. Existing search optimization methods can somewhat solve this issue, but they have limitations. Moreover, the calculation of the rule activation weight only considers the similarity between input data and a single rule, ignoring the influence of the rule linkage. To address these problems, we propose a new EBRB system based on the K-Nearest Neighbor graph index (Graph-EBRB). We introduce the Hierarchical Navigable Small World (HNSW) algorithm to create the K-Nearest Neighbor graph index of the EBRB system. This index allows us to efficiently search and activate a set of key rules. We also propose a new activation weight calculation method based on the Graph Convolution Neural Network (GCN), and we optimize the system performance using a parameter learning strategy. We conduct a comprehensive experiment on 14 commonly used public data sets, and the results show that Graph-EBRB system significantly improves the reasoning efficiency and accuracy of the EBRB system. Finally, we apply the Graph-EBRB system to tree disease identification and achieve excellent classification performance, identifying over 90% of the diseased trees on the complete dataset.

Similarity and dissimilarity relationships based graphs for multimodal change detection

Multimodal change detection (CD) is an increasingly interesting yet highly challenging subject in remote sensing. To facilitate the comparison of multimodal images, some image regression methods transform one image to the domain of the other image, allowing for images comparison in the same domain as in unimodal CD. In this paper, we begin by analyzing the limitations of previous image structure based regression models that only rely on similarity relationships. Then, we highlight the significance of incorporating dissimilarity relationships as a complementary approach to more comprehensively characterize and utilize the image structure. In light of this, we propose a novel method for multimodal CD called Similarity and Dissimilarity induced Image Regression (SDIR). Specifically, SDIR construct a similarity based k-nearest neighbors (KNN) graph and a dissimilarity based k-farthest neighbors (KFN) graph, where the former mainly characterizes the low-frequency information and the latter captures the high-frequency information in spectral domain. Therefore, the proposed SDIR that incorporates similarity (low-frequency) and dissimilarity (high-frequency) relationships enables to achieve better regression performance. After completing the image regression, we utilize a Markovian based fusion segmentation model to combine the change fusion and change extraction processes for improving the final CD accuracy. The proposed method’s effectiveness is demonstrated through experiments on six real datasets and compared with eleven advanced and widely used methods, achieving 5.6% improvements in the average Kappa coefficient. The source code is accessible at https://github.com/yulisun/SDIR.

A Kernel Measure of Dissimilarity between M Distributions

Given M ≥ 2 distributions defined on a general measurable space, we introduce a nonparametric (kernel) measure of multi-sample dissimilarity (KMD) — a parameter that quantifies the difference between the M distributions. The population KMD, which takes values between 0 and 1, is 0 if and only if all the M distributions are the same, and 1 if and only if all the distributions are mutually singular. Moreover, KMD possesses many properties commonly associated with f-divergences such as the data processing inequality and invariance under bijective transformations. The sample estimate of KMD, based on independent observations from the M distributions, can be computed in near linear time (up to logarithmic factors) using k-nearest neighbor graphs (for k ≥ 1 fixed). We develop an easily implementable test for the equality of M distributions based on the sample KMD that is consistent against all alternatives where at least two distributions are not equal. We prove central limit theorems for the sample KMD, and provide a complete characterization of the asymptotic power of the test, as well as its detection threshold. The usefulness of our measure is demonstrated via real and synthetic data examples; our method is also implemented in an R package.

Multi-level graph contrastive learning

Graph representation learning has attracted a surge of interest recently, which targets learning discriminant representation for each node in the graph. Most of these representation methods focus on supervised learning and heavily depend on label information. However, annotating graphs are expensive in the real-world, especially in specialized domains (i.e. biology), as it requires the annotators with the domain knowledge to label the graph. To approach this problem, self-supervised learning provides a feasible solution for graph representation learning. In this paper, we propose a Multi-Level Graph Contrastive Learning (MLGCL) framework for learning robust representation of graph data by contrasting space views of graphs. Specifically, we introduce a novel contrastive view — space view. The original graph is a first-order approximation structure in the topological space where nodes are linked by feature similarity, relationship, etc. While the k-nearest neighbor (kNN) graph with community structure generated by encoding features preserves high-order proximity in feature space, it not only provides a complementary graph to the original graph from the feature space view but also is suitable for GNNs encoder. Furthermore, we develop a multi-level contrastive mode to preserve the local similarity and semantic similarity of graph-structured data simultaneously. Extensive experiments indicate MLGCL achieves promising results compared with the existing state-of-the-art graph representation learning methods on seven node classification datasets and three graph classification datasets.