Adaptive K-nearest Neighbor Research Articles

Adaptive Graph [formula omitted]-Means

Clustering large-scale datasets has received increasing attention recently. However, existing algorithms are still not efficient in scenarios with extremely large number of clusters. To this end, Adaptive Graph K-Means (AGKM) is proposed in this work. Its idea originates from k-means, but it operates on an adaptive k-Nearest Neighbor (k-NN) graph instead of data features. First, AGKM is highly efficient for processing datasets where both the numbers of samples and clusters are very large. Specifically, the time and space complexity are both linear w.r.t the number of samples and, more importantly, independent to the cluster number. Second, AGKM is designed for balanced clusters. This constraint is realized by adding a regularization term in loss function, and a simple modification of the graph in optimization algorithm, which does not increase the computational burden. Last, the indicator and dissimilarity matrices are learned simultaneously, so that the proposed AGKM obtains the final partition directly with higher efficacy and efficiency. Experiments on several datasets validate the advantages of AGKM. In particular, over 29X and 46X speed-ups with respect to k-means are observed on the two large-scale datasets WebFace and CelebA, respectively.

ANNE: Adaptive Nearest Neighbours and Eigenvector-based sample selection for robust learning with noisy labels

An important stage of most state-of-the-art (SOTA) noisy-label learning methods consists of a sample selection procedure that classifies samples from the noisy-label training set into noisy-label or clean-label subsets. The process of sample selection typically consists of one of the two approaches: loss-based sampling, where high-loss samples are considered to have noisy labels, or feature-based sampling, where samples from the same class tend to cluster together in the feature space and noisy-label samples are identified as anomalies within those clusters. Empirically, loss-based sampling is robust to a wide range of noise rates, while feature-based sampling tends to work effectively in particular scenarios, e.g., the filtering of noisy instances via their eigenvectors (FINE) sampling exhibits greater robustness in scenarios with low noise rates, and the K nearest neighbour (KNN) sampling mitigates better high noise-rate problems. This paper introduces the Adaptive Nearest Neighbours and Eigenvector-based (ANNE) sample selection methodology, a novel approach that integrates loss-based sampling with the feature-based sampling methods FINE and Adaptive KNN to optimize performance across a wide range of noise rate scenarios. ANNE achieves this integration by first partitioning the training set into high-loss and low-loss sub-groups using loss-based sampling. Subsequently, within the low-loss subset, sample selection is performed using FINE, while the high-loss subset employs Adaptive KNN for effective sample selection. We integrate ANNE into the noisy-label learning state of the art (SOTA) method SSR+, and test it on CIFAR-10/-100 (with symmetric, asymmetric and instance-dependent noise), Webvision and ANIMAL-10, where our method shows better accuracy than the SOTA in most experiments, with a competitive training time. The code is available at https://github.com/filipe-research/anne.

Urban Land Use Classification Model Fusing Multimodal Deep Features

Urban land use classification plays a significant role in urban studies and provides key guidance for urban development. However, existing methods predominantly rely on either raster structure deep features through convolutional neural networks (CNNs) or topological structure deep features through graph neural networks (GNNs), making it challenging to comprehensively capture the rich semantic information in remote sensing images. To address this limitation, we propose a novel urban land use classification model by integrating both raster and topological structure deep features to enhance the accuracy and robustness of the classification model. First, we divide the urban area into block units based on road network data and further subdivide these units using the fractal network evolution algorithm (FNEA). Next, the K-nearest neighbors (KNN) graph construction method with adaptive fusion coefficients is employed to generate both global and local graphs of the blocks and sub-units. The spectral features and subgraph features are then constructed, and a graph convolutional network (GCN) is utilized to extract the node relational features from both the global and local graphs, forming the topological structure deep features while aggregating local features into global ones. Subsequently, VGG-16 (Visual Geometry Group 16) is used to extract the image convolutional features of the block units, obtaining the raster structure deep features. Finally, the transformer is used to fuse both topological and raster structure deep features, and land use classification is completed using the softmax function. Experiments were conducted using high-resolution Google images and Open Street Map (OSM) data, with study areas on the third ring road of Shenyang and the fourth ring road of Chengdu. The results demonstrate that the proposed method improves the overall accuracy and Kappa coefficient by 9.32% and 0.17, respectively, compared to single deep learning models. Incorporating subgraph structure features further enhances the overall accuracy and Kappa by 1.13% and 0.1. The adaptive KNN graph construction method achieves accuracy comparable to that of the empirical threshold method. This study enables accurate large-scale urban land use classification with reduced manual intervention, improving urban planning efficiency. The experimental results verify the effectiveness of the proposed method, particularly in terms of classification accuracy and feature representation completeness.

AKNNO: single-cell and spatial transcriptomics clustering with an optimized adaptive k-nearest neighbor graph

Typical clustering methods for single-cell and spatial transcriptomics struggle to identify rare cell types, while approaches tailored to detect rare cell types gain this ability at the cost of poorer performance for grouping abundant ones. Here, we develop aKNNO to simultaneously identify abundant and rare cell types based on an adaptive k-nearest neighbor graph with optimization. Benchmarking on 38 simulated and 20 single-cell and spatial transcriptomics datasets demonstrates that aKNNO identifies both abundant and rare cell types more accurately than general and specialized methods. Using only gene expression aKNNO maps abundant and rare cells more precisely compared to integrative approaches.

Adaptive three-way KNN classifier using density-based granular balls

The granular ball k-nearest neighbors (GBKNN) algorithm improves the efficiency and robustness of traditional k-nearest neighbors (KNN) by replacing point input with granular ball. However, in the generation process of granular balls by GBKNN, there may be unbalanced distribution of data points existed in some granular balls, which will cause more classification errors. In addition, the fixed value of k in GBKNN may also reduce the accuracy of classification. In order to address these issues, an adaptive three-way KNN classifier using density-based granular balls is proposed. Firstly, an improved density-based granular ball computing using density peak clustering is presented. This method introduces a refined threshold to subdivide the granular balls. Secondly, a data-driven neighborhood is defined to search the optimal k value and a density-based granular ball KNN (DBGBKNN) algorithm is proposed. Thirdly, by considering the fuzziness of the testing set in the classification process, the density-based granular ball KNN with three-way decision (DBGBKNN-3WD) is constructed. Finally, experimental results verify that DBGBKNN-3WD achieves high comprehensive score and low time complexity while maintaining less fuzziness loss than other algorithms.

Adaptive K values and training subsets selection for optimal K-NN performance on FPGA

This study introduces an Adaptive K-Nearest Neighbors methodology designed for FPGA platforms, offering substantial improvements over traditional K-Nearest Neighbors implementations. By integrating a dynamic classifier selection system, our approach enhances adaptability, enabling on-the-fly adjustments of K values and subsets of training data. This flexibility results in up to a 10.66% improvement in accuracy and significantly reduces latency, rendering our system up to 3.918 times more efficient than conventional K-Nearest Neighbors techniques. The methodology’s efficacy is validated through experiments across multiple datasets, demonstrating its potential in optimizing both classification accuracy and system efficiency. The adaptive approach’s ability to improve response times, along with its flexibility, positions it as an ideal solution for real-time applications and highlights the advantages of the adaptive K-Nearest Neighbors methodology in overcoming the constraints of hardware-accelerated machine learning.

Mechanical Fault Diagnosis of High-Voltage Circuit Breakers with Dynamic Multi-Attention Graph Convolutional Networks Based on Adaptive Graph Construction

With the rapid development of deep learning, its powerful capabilities make it possible to perform mechanical fault diagnosis of high-voltage circuit breakers (HVCBs). Among deep learning approaches, the convolutional neural network is widely used. However, while it can extract features effectively, it also has some limitations. Specifically, it depends on a large number of training data and only takes data information into account without considering structural information. These shortcomings lead to unused information and unsatisfactory model results. To address these shortcomings, this paper proposes AKNN-DMGCN, a novel dynamic multi-attention graph convolutional network based on an adaptively constructed graph, which can achieve high accuracy and robust mechanical fault diagnosis of HVCBs. First, a novel adaptive k-nearest neighbor (AKNN) graph construction method is proposed to construct informative graphs. The AKNN method can mine the relationship between the original data samples and utilize the data and label information. Thus, it has high fault tolerance to noise signals and can construct a structure graph with rich and accurate information, which can improve the overall model performance. Then, a dynamic multi-attention graph convolutional network (DMGCN) is applied for mechanical fault diagnosis of HVCBs. DMGCN fully utilizes structural and numerical information representing HVCB signals to perform classification. DMGCN has a dynamic multi-attention mechanism with strong expressive ability, which allows it to achieve high diagnostic accuracy. The experimental results indicate that the accuracy of AKNN-DMGCN reaches 97.22% on a balanced dataset and 95.01% on an imbalanced dataset, which demonstrates that the proposed method is effective for both balanced and imbalanced samples.

Major depressive disorder recognition by quantifying EEG signal complexity using proposed APLZC and AWPLZC

BackgroundSeeking objective quantitative indicators is important for accurately recognizing major depressive disorder (MDD). Lempel-Ziv complexity (LZC), employed to characterize neurological disorders, faces limitations in tracking dynamic changes in EEG signals due to defects in the coarse-graining process, hindering its precision for MDD objective quantitative indicators. MethodsThis work proposed Adaptive Permutation Lempel-Ziv Complexity (APLZC) and Adaptive Weighted Permutation Lempel-Ziv Complexity (AWPLZC) algorithms by refining the coarse-graining process and introducing weight factors to effectively improve the precision of LZC in characterizing EEGs and further distinguish MDD patients better. APLZC incorporated the ordinal pattern, while False Nearest Neighbor and Mutual Information algorithms were introduced to determine and adjust key parameters adaptively. Furthermore, we proposed AWPLZC by assigning different weights to each pattern based on APLZC. Thirty MDD patients and 30 healthy controls (HCs) were recruited and their 64-channel resting EEG signals were collected. The complexities of gamma oscillations were then separately computed using LZC, APLZC, and AWPLZC algorithms. Subsequently, a multi-channel adaptive K-nearest neighbor model was constructed for identifying MDD patients and HCs. ResultsLZC, APLZC, and AWPLZC algorithms achieved accuracy rates of 78.29 %, 90.32 %, and 95.13 %, respectively. Sensitivities reached 67.96 %, 85.04 %, and 98.86 %, while specificities were 88.62 %, 95.35 %, and 89.92 %, respectively. Notably, AWPLZC achieved the best performance in accuracy and sensitivity, with a specificity limitation. LimitationThe sample size is relatively small. ConclusionAPLZC and AWPLZC algorithms, particularly AWPLZC, demonstrate superior effectiveness in differentiating MDD patients from HCs compared with LZC. These findings hold significant clinical implications for MDD diagnosis.

Multiview Adaptive K-Nearest Neighbor Classification

Multiview Adaptive K-Nearest Neighbor Classification

An Efficient GNSS Coordinate Classification Strategy with an Adaptive KNN Algorithm for Epidemic Management

In times of widespread epidemics, numerous individuals are at risk of contracting viruses, such as COVID-19, monkeypox, and pneumonia, leading to a ripple effect of impacts on others. Consequently, the Centers for Disease Control (CDC) typically devises strategies to manage the situation by monitoring and tracing the infected individuals and their areas. For convenience, “targets” and “areas” represent the following individuals and areas. A global navigation satellite system (GNSS) can assist in evaluating the located areas of the targets with pointing-in-polygon (PIP) related technology. When there are many targets and areas, relying solely on PIP technology for classification from targets to areas could be more efficient. The classification technique of k-nearest neighbors (KNN) classification is widely utilized across various domains, offering reliable classification accuracy. However, KNN classification requires a certain quantity of targets with areas (training dataset) for execution, and the size of the training dataset and classification time often exhibit an exponential relationship. This study presents a strategy for applying KNN technology to classify targets into areas. Additionally, within the strategy, we propose an adaptive KNN algorithm to enhance the efficiency of the classification procedure.

Graph Distance and Adaptive K-Nearest Neighbors Selection-Based Density Peak Clustering

Graph Distance and Adaptive K-Nearest Neighbors Selection-Based Density Peak Clustering

Extended Collaborative Filtering Recommendation System with Adaptive KNN and SVD

In recent years, recommendation systems have gained significant importance due to the vast amount of digital content available on various online platforms. Collaborative filtering is a widely adopted approach in recommendation systems, leveraging user-item interactions to make personalized predictions. However, traditional collaborative filtering methods face challenges such as the cold-start problem and data sparsity. To address these issues, researchers have proposed advanced techniques, including Adaptive KNN-Based and SVD-Based Extended Collaborative Filtering. This paper provides a comprehensive review of these two recommendation systems, discussing their underlying principles, advantages, and limitations. Furthermore, we explore recent research advancements and real-world applications, providing insights into the potential future developments in this field.

A Novel Fingerprint Positioning Method Applying Vision-Based Definition for WIFI-Based Localization

Wireless Fidelity (WIFI) can transmit data efficiently, but it has the advantages of low computing cost and simple deployment. However, the WIFI signal is susceptible to interference from human movement, multipath propagation, and indoor temperature changes, resulting in problems such as low positioning accuracy and poor stability. To improve positioning accuracy and stability, we propose a novel fingerprint positioning technology applying vision-guided definition for WIFI-based localization, defined as KV. There are two stages in localization: WIFI-based coarse localization and fusion localization. In the WIFI-based coarse positioning stage, we propose a two-metric adaptive K-nearest Neighbor (KNN) localization method to improve the accuracy and robustness of WIFI-based localization. Unlike the traditional KNN method, a difference-based search approach is introduced to can obtain the K value intelligently and reasonably instead of manual determination. In the fusion localization stage, we employ an effective multi-angle unsupervised fusion positioning method applying the fusion of WIFI and vision to enhance positioning accuracy and positioning stability further. Experimental results show that our proposed fusion localization method achieves 1.24m in localization accuracy and the positioning error is 60% within 0.8 meters. The performance of the proposed method surmounts other state-of-the-art methods, which proves its effectiveness. This method has important practical significance in multi-source indoor positioning technology.

Extended Collaborative Filtering Recommendation System with Adaptive KNN and SVD

Extended Collaborative Filtering Recommendation System with Adaptive KNN and SVD

Structure-Aware Graph Convolution Network for Point Cloud Parsing

Point clouds are becoming a popular medium to describe 3D scenes, benefitting from their accuracy and completeness in expressing the spatial and geometrical information of objects. However, due to the disorder and uneven distribution nature, merely selecting neighbors for point clouds in Euclidean space is inefficient and position-ignoring. To fill this gap, we propose a structure-aware graph convolution network (SA-GCN), which consists of an adaptive dilated KNN module (ADKNN), a learnable graph filter (LGF), and a structure-aware feature transformation module (SFT). Specially, the ADKNN module can dynamically adjust the range of grouping neighbor points, while being universal to improve the performance of arbitrary KNN-based methods. Moreover, with the localized auxiliary information provided by LGF, our SFT module disentangles the spatial details as a sort of coding guidance for better deep feature representations. Extensive experimental results on point cloud classification and segmentation tasks demonstrate the superiority of our proposed network.

Adaptive KNN and graph-based auto-weighted multi-view consensus spectral learning

The multi-view learning is a fundamental problem in the multimedia analysis. However, most existing multi-view learning methods need to calculate a similarity matrix for each view. This operation has two drawbacks. Firstly, the construction of the similarity matrix needs a fixed function. But, in fact, the internal relationship between data points changes continuously. The fixed graph representation matrix is not conducive to fully consider the heterogeneous information about different views. Secondly, the similarity matrix of each view is used to participate independently in the subsequent calculation, which ignores the correlation and importance of different views. In our paper, an adaptive K Nearest Neighbor (KNN) and graph-based auto-weighted multi-view consensus spectral clustering (KGWMC) is proposed. An adaptive KNN method is used to calculate a similarity matrix for each view, which is updated in the subsequent iteration process. An adaptive joint optimization strategy is used to fuse multiple views for obtaining a consensus graph. In this step, instead of using a fixed function to calculate the weight of each view, we transform the weight calculation into an optimization problem for iteratively updating the weight of each view in a non-parametric form. The main novelty of KGWMC is the learning method of each similarity matrix and consensus graph in a mutual reinforcement manner. When the consensus graph converges, the spectral clustering algorithm is implemented to it for obtaining the final clustering. Finally, we demonstrate the effectiveness of our algorithm by comparing it with nine state-of-the-art algorithms on six real datasets.

Activity recognition on smartphones using an AKNN based support vectors

PurposeMobile phone-based human activity recognition (HAR) consists of inferring user’s activity type from the analysis of the inertial mobile sensor data. This paper aims to mainly introduce a new classification approach called adaptive k-nearest neighbors (AKNN) for intelligent HAR using smartphone inertial sensors with a potential real-time implementation on smartphone platform.Design/methodology/approachThe proposed method puts forward several modification on AKNN baseline by using kernel discriminant analysis for feature reduction and hybridizing weighted support vector machines and KNN to tackle imbalanced class data set.FindingsExtensive experiments on a five large scale daily activity recognition data set have been performed to demonstrate the effectiveness of the method in terms of error rate, recall, precision, F1-score and computational/memory resources, with several comparison with state-of-the art methods and other hybridization modes. The results showed that the proposed method can achieve more than 50% improvement in error rate metric and up to 5.6% in F1-score. The training phase is also shown to be reduced by a factor of six compared to baseline, which provides solid assets for smartphone implementation.Practical implicationsThis work builds a bridge to already growing work in machine learning related to learning with small data set. Besides, the availability of systems that are able to perform on flight activity recognition on smartphone will have a significant impact in the field of pervasive health care, supporting a variety of practical applications such as elderly care, ambient assisted living and remote monitoring.Originality/valueThe purpose of this study is to build and test an accurate offline model by using only a compact training data that can reduce the computational and memory complexity of the system. This provides grounds for developing new innovative hybridization modes in the context of daily activity recognition and smartphone-based implementation. This study demonstrates that the new AKNN is able to classify the data without any training step because it does not use any model for fitting and only uses memory resources to store the corresponding support vectors.

Weighted Adaptive KNN Algorithm With Historical Information Fusion for Fingerprint Positioning

<inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">${K}$ </tex-math></inline-formula> -nearest neighbors (KNN) algorithms are widely used for indoor fingerprint positioning, but conventional KNN algorithms usually adopt received signal strength (RSS) similarity as a metric to select <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">${K}$ </tex-math></inline-formula> reference points (RPs) for position determination, which may lead to inaccurate positioning results. This is because RSS similarity cannot well reflect position proximity due to the exponential relationship of RSS and propagation distance. To address these issues, this letter proposes a novel weighted adaptive KNN algorithm with historical information fusion for fingerprint positioning, which can choose a variable number of RPs according to both the improved RSS similarity and position proximity. Particularly, useful historical information extracted from a trajectory is used to improve further positioning accuracy. Finally, experiments are conducted on two different databases and the results validate performance of the proposed algorithm.

A new method to build the adaptive k-nearest neighbors similarity graph matrix for spectral clustering

In spectral clustering (SC), the clustering result highly depends on the similarity graph matrix. The k-nearest neighbors graph is a popular method to build the similarity graph matrix with a sparse structure for better graph cutting. However, many current methods require that the parameter k is specified by the user, and specifying an appropriate k for an unknown data set is often a difficult task. In this paper, we propose a new method for building the adaptive k-nearest neighbors similarity graph (AKNNG). The AKNNG specifies different k values for different data points to obtain a better graph structure. Specifically, it sets a maximum number of the nearest neighbors kmax and assigns a different k value (k⩽kmax) for each data point. The k value is adjusted automatically by cutting some weak connections from each data point according to the m powers transform of the similarity graph. The experimental results on Spiral, Multi-clusters, Yale and Coil20 datasets have shown that when setting kmax=20, the new method has improved the clustering accuracies of these four datasets over 4%, 6%, 4%, 5%, respectively, in comparison with those by the existing methods. The new method can also reduce the sensitiveness of the number of nearest neighbors, and build the similarity graph with less time.

Binary Grey Wolf Optimizer with Mutation and Adaptive K-nearest Neighbour for Feature Selection in Parkinson’s Disease Diagnosis

Disease identification and classification relies on Feature Selection (FS) to find the relevant features for accurate medical diagnosis. FS is an optimization problem solved with the help of stochastic optimization strategies. Research on selection of relevant features in Parkinson’s disease is challenging due to the lack of enough information about the disease. In this work, a wrapper-based Binary Improved Grey Wolf Optimizer (BIGWO) approach is developed for categorizing the Parkinson’s disease with optimal set of features. The proposed approach employs five different transfer functions to encode the search space of features. A mutation operation in BIGWO fine tunes the search process to determine the best features for disease diagnosis. Further, the number of neighbours in k-Nearest Neighbour (kNN) is optimized using the Adaptive kNN (AkNN) in the fitness evaluation of the BIGWO algorithm which improves classification performance. Different variants of the proposed BIGWO algorithm are compared with other algorithms to assess their performance. Experiments are conducted on four benchmark datasets that demonstrate the two variant BIGWO-V1 and BIGWO-V2 algorithms outperform other four meta-heuristic (GA, PSO, BBA, and MCS) algorithms.