K-nearest Neighbor Graph Research Articles

NonPC: Non-parametric clustering algorithm with adaptive noise detecting

Graph-based clustering performs efficiently for identifying clusters in local and nonlinear data Patterns. The existing methods face the problem of parameter selection, such as the setting of k of the k-nearest neighbor graph and the threshold in noise detection. In this paper, a non-parametric clustering algorithm (NonPC) is proposed to tackle those inherent limitations and improve clustering performance. The weighted natural neighbor graph (wNaNG) is developed to represent the given data without any prior knowledge. What is more, the proposed NonPC method adaptively detects noise data in an unsupervised way based on some attributes extracted from wNaNG. The algorithm works without preliminary parameter settings while automatically identifying clusters with unbalanced densities, arbitrary shapes, and noises. To assess the advantages of the NonPC algorithm, extensive experiments have been conducted compared with some classic and recent clustering methods. The results demonstrate that the proposed NonPC algorithm significantly outperforms the state-of-the-art and well-known algorithms in Adjusted Rand index, Normalized Mutual Information, and Fowlkes-Mallows index aspects.

Constrained Density Peak Clustering

Clustering is a commonly used tool for discovering knowledge in data mining. Density peak clustering (DPC) has recently gained attention for its ability to detect clusters with various shapes and noise, using just one parameter. DPC has shown advantages over other methods, such as DBSCAN and K-means, but it struggles with datasets that have both high and low-density clusters. To overcome this limitation, the paper introduces a new semi-supervised DPC method that improves clustering results with a small set of constraints expressed as must-link and cannot-link. The proposed method combines constraints and a k-nearest neighbor graph to filter out peaks and find the center for each cluster. Constraints are also used to support label assignment during the clustering procedure. The efficacy of this method is demonstrated through experiments on well-known data sets from UCI and benchmarked against contemporary semi-supervised clustering techniques.

Random projection forest initialization for graph convolutional networks

Graph convolutional networks (GCNs) were a great step towards extending deep learning to graphs. GCN uses the graph G and the feature matrix X as inputs. However, in most cases the graph G is missing and we are only provided with the feature matrix X. To solve this problem, classical graphs such as k-nearest neighbor (k-nn) are usually used to construct the graph G and initialize the GCN. Although it is computationally efficient to construct k-nn graphs, the constructed graph might not be very useful for learning. In a k-nn graph, points are restricted to have a fixed number of edges, and all edges in the graph have equal weights. Our contribution is Initializing GCN using a graph with varying weights on edges, which provides better performance compared to k-nn initialization. Our proposed method is based on random projection forest (rpForest). rpForest enables us to assign varying weights on edges indicating varying importance, which enhanced the learning. The number of trees is a hyperparameter in rpForest. We performed spectral analysis to help us setting this parameter in the right range. In the experiments, initializing the GCN using rpForest provides better results compared to k-nn initialization.•Constructing the graph G using rpForest sets varying weights on edges, which represents the similarity between a pair of samples.Unlike k-nearest neighbor graph where all weights are equal.•Using rpForest graph to initialize GCN provides better results compared to k-nn initialization. The varying weights in rpForest graph quantify the similarity between samples, which guided the GCN training to deliver better results.•The rpForest graph involves the tuning of the hyperparameter (number of trees T). We provided an informative way to set this hyperparameter through spectral analysis.

A spatiotemporal dissolved oxygen prediction model based on graph attention networks suitable for missing data.

The accurate prediction of dissolved oxygen concentration is crucial for the effective prevention and control of water pollution. A spatiotemporal prediction model for dissolved oxygen content that is suitable for missing data is proposed in this study. The model utilizes a module based on neural controlled differential equations (NCDEs) to handle missing data and graph attention networks (GATs) to capture the spatiotemporal relationship of dissolved oxygen content. To enhance the performance of model, it is optimized from three aspects: an iterative optimization method based on the k-nearest neighbor graph is proposed to enhance the quality of graph; Shapley additive explanations model (SHAP) is used to select the main features into model, enabling it to handle multiple features; and a fusion graph attention mechanism is introduced to improve the robustness of model to noise. The model is evaluated using data from water quality monitoring sites in Hunan Province, China, from January 14, 2021, to June 16, 2022. The proposed model outperforms other models in long-term prediction (step = 18), with MAE of 0.194, NSE of 0.914, RAE of 0.219, and IA of 0.977. The results demonstrate that constructing appropriate spatial dependencies can enhance the accuracy of dissolved oxygen prediction models, and the NCDE module confers robustness to missing data in the model.

GCANet: Geometry cues-aware facial expression recognition based on graph convolutional networks

Facial expression recognition (FER) task in the wild is challenging due to some uncertainties, such as the ambiguity of facial expressions, subjective annotations, and low-quality facial images. A novel model for FER in-the-wild datasets is proposed in this study to solve these uncertainties. The overview of the proposed method is as follows. First, the facial images are grouped into high and low uncertainties by the pre-trained network. The graph convolutional network (GCN) framework is then used for the facial images with low uncertainty to obtain geometry cues, including the relationship among action units (AUs) and the implicit connection between AUs and expressions, which help predict the probability of the underlying emotional label. The emotion label distribution is produced by combining the predicted latent label probability and the given label. For the facial images with high uncertainty, k-nearest neighbor graphs are built to determine the k facial images in the low uncertainty group with the highest similarity to the given facial image. The emotion label distribution of the given image is then replaced by fusing the emotion label distribution based on the distances between the given image and its adjacent images. Finally, the constructed emotion label distribution facilitates training in a straightforward manner using a convolutional neural network framework to identify facial expressions. Experimental results on RAF-DB, FERPlus, AffectNet, and SFEW2.0 datasets demonstrate that the proposed method achieved superior performance compared to state-of-the-art approaches.

Generalized Stereo Matching Method Based on Iterative Optimization of Hierarchical Graph Structure Consistency Cost for Urban 3D Reconstruction

Generalized stereo matching faces the radiation difference and small ground feature difference brought by different satellites and different time phases, while the texture-less and disparity discontinuity phenomenon seriously affects the correspondence between matching points. To address the above problems, a novel generalized stereo matching method based on the iterative optimization of hierarchical graph structure consistency cost is proposed for urban 3D scene reconstruction. First, the self-similarity of images is used to construct k-nearest neighbor graphs. The left-view and right-view graph structures are mapped to the same neighborhood, and the graph structure consistency (GSC) cost is proposed to evaluate the similarity of the graph structures. Then, cross-scale cost aggregation is used to adaptively weight and combine multi-scale GSC costs. Next, object-based iterative optimization is proposed to optimize outliers in pixel-wise matching and mismatches in disparity discontinuity regions. The visibility term and the disparity discontinuity term are iterated to continuously detect occlusions and optimize the boundary disparity. Finally, fractal net evolution is used to optimize the disparity map. This paper verifies the effectiveness of the proposed method on a public US3D dataset and a self-made dataset, and compares it with state-of-the-art stereo matching methods.

Reconstruction of Single-Cell Trajectories Using Stochastic Tree Search.

The recent advancement in single-cell RNA sequencing technologies enables the understanding of dynamic cellular processes at the single-cell level. Using trajectory inference methods, pseudotimes can be estimated based on reconstructed single-cell trajectories which can be further used to gain biological knowledge. Existing methods for modeling cell trajectories, such as minimal spanning tree or k-nearest neighbor graph, often lead to locally optimal solutions. In this paper, we propose a penalized likelihood-based framework and introduce a stochastic tree search (STS) algorithm aiming at the global solution in a large and non-convex tree space. Both simulated and real data experiments show that our approach is more accurate and robust than other existing methods in terms of cell ordering and pseudotime estimation.

Centerless Clustering.

Although lots of clustering models have been proposed recently, k-means and the family of spectral clustering methods are both still drawing a lot of attention due to their simplicity and efficacy. We first reviewed the unified framework of k-means and graph cut models, and then proposed a clustering method called k-sums where a k-nearest neighbor ( k-NN) graph is adopted. The main idea of k-sums is to minimize directly the sum of the distances between points in the same cluster. To deal with the situation where the graph is unavailable, we proposed k-sums-x that takes features as input. The computational and memory overhead of k-sums are both O(nk), indicating that it can scale linearly w.r.t. the number of objects to group. Moreover, the costs of computational and memory are Irrelevant to the product of the number of points and clusters. The computational and memory complexity of k-sums-x are both linear w.r.t. the number of points. To validate the advantage of k-sums and k-sums-x on facial datasets, extensive experiments have been conducted on 10 synthetic datasets and 17 benchmark datasets. While having a low time complexity, the performance of k-sums is comparable with several state-of-the-art clustering methods.

A Graph-Based Blocking Approach for Entity Matching Using Contrastively Learned Embeddings

Data integration is considered a crucial task in the entity matching process. In this process, redundant and cunning entries must be identified and eliminated to improve the data quality. To archive this, a comparison between all entities is performed. However, this has quadratic computational complexity. To avoid this, `blocking' limits comparisons to probable matches. This paper presents a k-nearest neighbor graph-based blocking approach utilizing state-of-the-art context-aware sentence embeddings from pre-trained transformers. Our approach maps each database tuple to a node and generates a graph where nodes are connected by edges if they are related. We then invoke unsupervised community detection techniques over this graph and treat blocking as a graph clustering problem. Our work is motivated by the scarcity of training data for entity matching in real-world scenarios and the limited scalability of blocking schemes in the presence of proliferating data. Additionally, we investigate the impact of contrastively trained embeddings on the above system and test its capabilities on four data sets exhibiting more than 6 million comparisons. We show that our block processing times on the target benchmarks vary owing to the efficient data structure of the k-nearest neighbor graph. Our results also show that our method achieves better performance in terms of F1 score when compared to current deep learning-based blocking solutions.

The manifold regularized SVDD for noisy label detection

Supervised Learning (SL) celebrates a lot of research topics in machine learning (ML) and provides a large number of applications in multimedia. The effectiveness and performance of supervised learning are highly dependent on the quality of labeled samples, and poor or mislabeled labels always damage pre-knowledge and produce disastrous results. One of the notable works is the development of a model for detecting noisy labels in large-scale data sets for supervised classification. This paper introduces a new method for noisy label detection that combines the support vector data description (SVDD) model with manifold regularization. The proposed method is able to detect the noisy label in the testing set. In addition, we adopt K nearest neighbors to obtain the K-nearest neighbor graph, and the k-core algorithm is used to select noise nodes by selecting the influential nodes with larger k values in the K-NN graph. Experimental results over UCI data sets show that the manifold regularized SVDD can effectively detect the noise labels data, and presents better performance over most methods in terms of the outlier detection accuracy, and the average accuracy is mostly above 50%, even 100%. Moreover, manifold regularized SVDD has good performance in detecting the most influential top-k nodes compared with other methods.

A study on the effects of normalized TSP features for automated algorithm selection

Classic automated algorithm selection (AS) for (combinatorial) optimization problems heavily relies on so-called instance features, i.e., numerical characteristics of the problem at hand ideally extracted with computationally low-demanding routines. For the traveling salesperson problem (TSP) a plethora of features have been suggested. Most of these features are, if at all, only normalized imprecisely raising the issue of feature values being strongly affected by the instance size. Such artifacts may have detrimental effects on algorithm selection models. We propose a normalization for two feature groups which stood out in multiple AS studies on the TSP: (a) features based on a minimum spanning tree (MST) and (b) nearest neighbor relationships of the input instance. To this end we theoretically derive minimum and maximum values for properties of MSTs and k-nearest neighbor graphs (NNG) of Euclidean graphs. We analyze the differences in feature space between normalized versions of these features and their unnormalized counterparts. Our empirical investigations on various TSP benchmark sets point out that the feature scaling succeeds in eliminating the effect of the instance size. A proof-of-concept AS-study shows promising results: models trained with normalized features tend to outperform those trained with the respective vanilla features.

Graph-based discriminative features learning for fine-grained image retrieval

Fine-grained image retrieval has gradually become a hot topic in computer vision , which aims to retrieve images with the same subcategories from general visual categories. Though fine-grained image retrieval has made a breakthrough with the development of convolutional neural networks, its performance is still limited by the low discriminative feature embedding. To solve this problem, most prior works focus on mining more discriminative features with various strategies. In this paper, we propose a novel graph-based discriminative features learning network for fine-grained image retrieval (GDF-Net). We first design a global fine-grained feature aggregation module, which reconstructs the discriminative features through capturing context correlation based on a K-Nearest Neighbor graph. To reduce storage overhead and speed up retrieval, we further design a semantic hash encoding module, which generates a semantically compact hash code under the guidance of Cauchy quantization loss and bit balance loss. Validated by extensive experiments and ablation studies, our method consistently outperforms state-of-the-art generic retrieval methods as well as fine-grained retrieval methods on three datasets, e.g., CUB Birds, Stanford Dogs and Stanford Cars.

Improving Cluster-Based Index Structure for Approximate Nearest Neighbor Graph Search by Deep Learning-Based Hill-Climbing

This study presents a novel approach to archive an excellent tradeoff between search accuracy and computation cost in approximate nearest neighbor search. Usually, the k-nearest neighbor (kNN) graph and hill-climbing algorithm are adopted to accelerate the search process. However, using random seeds in the original hill-climbing is inefficient as they initiate an unsuitable search with inappropriate sources. Instead, we propose a neural network model to generate high-quality seeds that can boost query assignment efficiency. We evaluated the experiment on the benchmarks of SIFT1M and GIST1M datasets and showed the proposed seed prediction model effectively improves the search performance.

A split–merge clustering algorithm based on the k-nearest neighbor graph

Numerous graph-based clustering algorithms relying on k-nearest neighbor (KNN) have been proposed. However, the performance of these algorithms tends to be affected by many factors such as cluster shape, cluster density and outliers. To address these issues, we present a split–merge clustering algorithm based on the KNN graph (SMKNN), which is based on the idea that two adjacent clusters can be merged if the data points located in the connection layers of the two clusters tend to be consistent in distribution. In Stage 1, a KNN graph is constructed. In Stage 2, the subgraphs are obtained by removing the pivot points from the KNN graph, in which the pivot points are determined by the size of local distance ratio of data points. In Stage 3, the adjacent cluster pairs satisfying the maximum similarity are merged, in which the similarity measure of two clusters is designed with two concepts including external connection edges and internal connection edges. By the experiments on ten synthetic data sets and eight real data sets, we compared SMKNN with two traditional algorithms, two density-based algorithms, nine graph-based algorithms and four neural network based algorithms in accuracy. The experimental results demonstrate a good performance of the proposed clustering method.

A robust clustering method with noise identification based on directed K-nearest neighbor graph

Obtaining the optimal cluster number and generating reliable clustering results in nonlinear manifolds are necessary but challenging tasks. Most existing clustering algorithms have considerable limitations in dealing with local and nonlinear data patterns, while graph-based clustering has shown impressive performance in identifying clusters in such data patterns. In this paper, we propose a robust clustering method with noise cutting based on directed k-nearest neighbor graph (CDKNN) to identify the desired cluster number automatically and produce reliable clustering results simultaneously on nonlinear, non-overlapping but locally tight-connected data patterns. This method draws support from the k-nearest neighbor graph to represent the complex nonlinear datasets and applies parameter adaptive process to make the proposed clustering method better adapt to specific data patterns. The proposed method is robust to the noises of arbitrary shape datasets because it uses a directed K-nearest neighbor to cut out sparse nodes. We use simulation and UCI real-world datasets to prove the validity of the innovatory method by comparing it to k-means, DBSCAN, OPTICS, AP, SC, and CutPC algorithms in terms of clustering ACC, ARI, NMI, and FMI. The experimental results confirm that the proposed method outperforms the alternative nonlinear clustering methods.

All-Cause Death Prediction Method for CHD Based on Graph Convolutional Networks

Coronary heart disease (CHD) has become one of the most serious public health issues due to its high morbidity and mortality rates. Most of the existing coronary heart disease risk prediction models manually extract features based on shallow machine learning methods. It only focuses on the differences between local patient features and ignores the interaction modeling between global patients. Its accuracy is still insufficient for individualized patient management strategies. In this paper, we propose CHD prediction as a graph node classification task for the first time, where nodes can represent individuals in potentially diseased populations and graphs intuitively represent associations between populations. We used an adaptive multi-channel graph convolutional neural network (AM-GCN) model to extract graph embeddings from topology, node features, and their combinations through graph convolution. Then, the adaptive importance weights of the extracted embeddings are learned by using an attention mechanism. For different situations, we model the relationship of the CHD population with the population graph and the K-nearest neighbor graph method. Our experimental evaluation explored the impact of the independent components of the model on the CHD disease prediction performance and compared it to different baselines. The experimental results show that our new model exhibits the best experimental results on the CHD dataset, with a 1.3% improvement in accuracy, a 5.1% improvement in AUC, and a 4.6% improvement in F1-score compared to the nongraph model.

EEG-GAT: Graph Attention Networks for Classification of Electroencephalogram (EEG) Signals.

Graph neural networks (GNN) are an emerging framework in the deep learning community. In most GNN applications, the graph topology of data samples is provided in the dataset. Specifically, the graph shift operator (GSO), which could be adjacency, graph Laplacian, or their normalizations, is known a priori. However we often have no knowledge of the grand-truth graph topology underlying real-world datasets. One example of this is to extract subject-invariant features from physiological electroencephalogram (EEG) to predict a cognitive task. Previous methods use electrode sites to represent a node in the graph and connect them in various ways to hand-engineer a GSO e.g., i) each pair of electrode sites is connected to form a complete graph, ii) a specific number of electrode sites are connected to form a k-nearest neighbor graph, iii) each pair of electrode site is connected only if the Euclidean distance is within a heuristic threshold. In this paper, we overcome this limitation by parameterizing the GSO using a multi-head attention mechanism to explore the functional neural connectivity subject to a cognitive task between different electrode sites, and simultaneously learn the unsupervised graph topology in conjunction with the parameters of graph convolutional kernels.

Component‐based nearest neighbour subspace clustering

In this paper, the problem of clustering data points that lie near or on a union of independent low-dimensional subspaces is addressed. To this end, the popular spectral clustering-based algorithms usually follow a two-stage strategy that initially builds an affinity matrix and then applies spectral clustering. However, an inappropriate affinity matrix that does not sufficiently connect data points lying on the same subspace will easily lead to the issue of over-segmentation. To alleviate this issue, building the affinity matrix based on subspace hypotheses generated by an iterative sampling operation according to the Random Cluster Model under the framework of energy minimisation is proposed. Specifically, each hypothesis is generated from a large number of data points by sampling a component in a K-nearest neighbour graph. Extensive experiments on synthetic data and real-world datasets show that the proposed method can improve the connectivity of the affinity matrix and provide competitive results against state-of-the-art methods.

A new method to build the adaptive k-nearest neighbors similarity graph matrix for spectral clustering

In spectral clustering (SC), the clustering result highly depends on the similarity graph matrix. The k-nearest neighbors graph is a popular method to build the similarity graph matrix with a sparse structure for better graph cutting. However, many current methods require that the parameter k is specified by the user, and specifying an appropriate k for an unknown data set is often a difficult task. In this paper, we propose a new method for building the adaptive k-nearest neighbors similarity graph (AKNNG). The AKNNG specifies different k values for different data points to obtain a better graph structure. Specifically, it sets a maximum number of the nearest neighbors kmax and assigns a different k value (k⩽kmax) for each data point. The k value is adjusted automatically by cutting some weak connections from each data point according to the m powers transform of the similarity graph. The experimental results on Spiral, Multi-clusters, Yale and Coil20 datasets have shown that when setting kmax=20, the new method has improved the clustering accuracies of these four datasets over 4%, 6%, 4%, 5%, respectively, in comparison with those by the existing methods. The new method can also reduce the sensitiveness of the number of nearest neighbors, and build the similarity graph with less time.

Triku: a feature selection method based on nearest neighbors for single-cell data.

BackgroundFeature selection is a relevant step in the analysis of single-cell RNA sequencing datasets. Most of the current feature selection methods are based on general univariate descriptors of the data such as the dispersion or the percentage of zeros. Despite the use of correction methods, the generality of these feature selection methods biases the genes selected towards highly expressed genes, instead of the genes defining the cell populations of the dataset.ResultsTriku is a feature selection method that favors genes defining the main cell populations. It does so by selecting genes expressed by groups of cells that are close in the k-nearest neighbor graph. The expression of these genes is higher than the expected expression if the k-cells were chosen at random. Triku efficiently recovers cell populations present in artificial and biological benchmarking datasets, based on adjusted Rand index, normalized mutual information, supervised classification, and silhouette coefficient measurements. Additionally, gene sets selected by triku are more likely to be related to relevant Gene Ontology terms and contain fewer ribosomal and mitochondrial genes.ConclusionTriku is developed in Python 3 and is available at https://github.com/alexmascension/triku.