Nearest-neighbor Classifier Research Articles

Three-way decision-based noise correction for crowdsourcing

In crowdsourcing learning, a certain degree of label noise still exists in the integrated labels, which are obtained by label integration algorithms. In order to decrease the impact of label noise, many scholars focus on noise correction algorithms. Existing noise correction algorithms usually divide a data set with integrated labels into a clean set and a noise set, directly. However, there are certain noise instances in the clean set and clean instances in the noise set. To reduce the interference from noise instances and improve the utilization of clean instances, inspired by the three-way decision theory, we propose a three-way decision-based noise correction (TDNC) algorithm. Firstly, we calculate the compactness of each instance's multiple noisy label set and estimate the label certainty of each instance's integrated label. Secondly, we divide a data set with integrated labels into three disjoint sets, called positive set, boundary set and negative set, respectively. Thirdly, we further divide and add the instances in the boundary set into the positive set and the negative set according to the nearest neighbor classification. Finally, we train two heterogeneous classifiers on the new positive set to correct the noise instances in the new negative set using a consensus voting strategy. The experimental results on 34 simulated and two real-world data sets consistently show that TDNC significantly outperforms the existing state-of-the-art noise correction algorithms in terms of the noise ratio.

Hybrid machine learning techniques in the management of harmful algal blooms impact

Harmful algal blooms (HABs) are episodes of high concentrations of algae that are potentially toxic for human consumption. Mollusc farming can be affected by HABs because, as filter feeders, they can accumulate high concentrations of marine biotoxins in their tissues. To avoid the risk to human consumption, harvesting is prohibited when toxicity is detected. At present, the closure of production areas is based on expert knowledge and the existence of a predictive model would help when conditions are complex and sampling is not possible. Although the concentration of toxin in meat is the method most commonly used by experts in the control of shellfish production areas, it is rarely used as a target by automatic prediction models. This is largely due to the irregularity of the data due to the established sampling programs. As an alternative, the activity status of production areas has been proposed as a target variable based on whether mollusc meat has a toxicity level below or above the legal limit. This new option is the most similar to the actual functioning of the control of shellfish production areas. For this purpose, we have made a comparison between hybrid machine learning models like Neural-Network-Adding Bootstrap (BAGNET) and Discriminative Nearest Neighbor Classification (SVM-KNN) when estimating the state of production areas. The study has been carried out in several estuaries with different levels of complexity in the episodes of algal blooms to demonstrate the generalization capacity of the models in bloom detection. As a result, we could observe that, with an average recall value of 93.41% and without dropping below 90% in any of the estuaries, BAGNET outperforms the other models both in terms of results and robustness.

P3DC-shot: Prior-driven discrete data calibration for nearest-neighbor few-shot classification

Nearest-Neighbor (NN) classification has been proven as a simple and effective approach for few-shot learning. The query data can be classified efficiently by finding the nearest support class based on features extracted by pre-trained deep models. However, NN-based methods are sensitive to the data distribution and may produce false prediction if the samples in the support set happen to lie around the distribution boundary of different classes. To solve this issue, we present P3DC-shot, an improved nearest-neighbor based few-shot classification method empowered by prior-driven data calibration. Inspired by the distribution calibration technique which utilizes the distribution or statistics of the base classes to calibrate the data for few-shot tasks, we propose a novel discrete data calibration operation which is more suitable for NN-based few-shot classification. Specifically, we treat the prototypes representing each base class as priors and calibrate each support data based on its similarity to different base prototypes. Then, we perform NN classification using these discretely calibrated support data. Results from extensive experiments on various datasets show our efficient non-learning based method can outperform or at least comparable to SOTA methods which need additional learning steps.

Monitoring Software Execution Flow Through Power Consumption and Dynamic Time Warping

This work presents a technique for non-intrusive code execution tracking using side-channel signals of power consumption. Using a nearest-neighbor classifier that integrates the Dynamic Time Warping distance with information from the Control Flow Graph, it is possible to identify executed basic blocks from a trace of power consumption that exhibits temporal distortions due to assembly-level artifacts and varying operational conditions. Experimental results show that the proposed technique achieves over 95% precision when inferring the runtime execution flow of a cruise control application using unmarked traces of power consumption collected from different processors.

Novel Robust On-Line Indoor Occupancy Counting System Using mmWave Radar

In this work, a novel robust on-line indoor occupancy counting approach is proposed using the millimeter wave (mmWave) frequency-modulated continuous-wave (FMCW) radar. The acquired radar data can be represented as sparse three-dimensional (3D) radar point-clouds. The conventional indoor occupancy-counting schemes suffer from various types of errors and uncertainties emerging from mmWave radar sensors. Thus, we propose a novel feature-extraction strategy to obtain the robust features for training the machine-learning driven occupancy-counter. We formulate the underlying indoor occupancy-counting problem as the multi-classification problem such that the k nearest neighbor (KNN) classifier is adopted to identify the number of occupants on line. Experimental results from realworld data demonstrate that our proposed new approach leads to a promising classification-accuracy of 95.8% for indoor occupancy counting. In the comparative study based on the realworld data, our proposed novel indoor occupancy-counting method greatly outperforms other existing schemes.

INVESTIGATION OF DIFFERENT MACHINE LEARNING CLASSIFIERS FOR TIMELY FORECAST OF BREAST CANCER USING WISCONSIN DATASET

The main aims, methods, and results of the study on breast cancer detection using various machine learning classifiers. It seems that the study focused on analyzing the performance of different classifiers such as Logistic Regression, KNN, SVM-LC, SBM-RBF, Gaussian Naïve Bayes, Decision Tree, and Random Forest Classifier on the Wisconsin dataset. The study aimed to measure the accuracy of these classifiers in detecting breast cancer at an early stage. The Wisconsin dataset is a well-known dataset frequently used for breast cancer research and contains relevant features for classification. According to the testing accuracy results you provided, each classifier achieved the following accuracy scores: Logistic Regression=0.9440, K Nearest Neighbor=0.9580, Support Vector Machine (Linear Classifier) =0.9650, Support Vector Machine (RBF Classifier) =0.9650, Gaussian Naïve Bayes=0.9230, Decision Tree=0.9510 and Random Forest Classifier=0.9650. Based on these accuracy outcomes, it can be concluded that the proposed machine learning models, particularly Support Vector Machines (both linear and RBF), as well as K Nearest Neighbor and Random Forest Classifier, performed well in classifying breast cancer using the Wisconsin dataset. Logistic Regression, Decision Tree, and Gaussian Naïve Bayes also achieved reasonably good accuracy scores. The study suggests that the proposed models have the potential to assist medical professionals in accurately classifying breast lesions, which can lead to early detection and better management of breast cancer.

Online Evidential Nearest Neighbour Classification for Internet of Things Time Series

SummaryThe ‘Internet of Things’ (IoT) is a rapidly developing set of technologies that leverages large numbers of networked sensors, to relay data in an online fashion. Typically, knowledge of the sensor environment is incomplete and subject to changes over time. There is a need to employ classification algorithms to understand the data. We first review of existing time series classification (TSC) approaches, with emphasis on the well‐known k‐nearest neighbours (kNN) methods. We extend these to dynamical kNN classifiers, and discuss their shortcomings for handling the inherent uncertainty in IoT data. We next review evidential kNN ( ) classifiers that leverage the well‐known Dempster–Shafer theory to allow principled uncertainty quantification. We develop a dynamic approach for classifying IoT streams via algorithms that use evidential theoretic pattern rejection rules for (i) classifying incoming patterns into a set of oracle classes, (ii) automatically pruning ambiguously labelled patterns such as aberrant streams (due to malfunctioning sensors, say), and (iii) identifying novel classes that may emerge in new subsequences over time. While these methods have wide applicability in many domains, we illustrate the dynamic and approaches for classifying a large, noisy IoT time series dataset from an insurance firm.

A prototype selection technique based on relative density and density peaks clustering for k nearest neighbor classification

k-nearest neighbor classifier (KNN) is one of the most famous classification models due to its straightforward implementation and an error bounded by twice the Bayes error. However, it usually degrades because of noise and the high cost in computing the distance between different samples. In this context, hybrid prototype selection techniques have been postulated as a good solution and developed. Yet, they have the following issues: (a) adopted edition methods are susceptible to harmful samples around tested samples; (b) they retain too many internal samples, which contributes little to the classification of KNN classifier and (or) leading to the low reduction; (c) they rely on many parameters. The main contributions of our work are that (a) a novel competitive hybrid prototype selection technique based on relative density and density peaks clustering (PST-RD-DP) are proposed against the above issues at the same time; (b) a new edition method based on relative density and distance (EMRDD) in PST-RD-DP is first proposed to remove harmful samples and smooth the class boundary; (c) a new condensing method based on relative density and density peaks clustering (CMRDDPC) in PST-RD-DP is second proposed to retain representative borderline samples. Intensive experiments prove that PST-RD-DP outperforms 6 popular hybrid prototype selection techniques on extensive real data sets in weighing accuracy and reduction of the KNN classifier. Besides, the running time of PST-RD-DP is also acceptable.

Enriching Representation and Enhancing Nearest Neighbor Classification of Slope/Landslide Data Using Rectified Feature Line Segments and Hypersphere-Based Scaling: A Reproducible Experimental Comparison

Measuring geotechnical and natural hazard engineering features, along with pattern recognition algorithms, allows us to categorize the stability of slopes into two main classes of interest: stable or at risk of collapse. The problem of slope stability can be further generalized to that of assessing landslide susceptibility. Many different methods have been applied to these problems, ranging from simple to complex, and often with a scarcity of available data. Simple classification methods are preferred for the sake of both parsimony and interpretability, as well as to avoid drawbacks such as overtraining. In this paper, an experimental comparison was carried out for three simple but powerful existing variants of the well-known nearest neighbor rule for classifying slope/landslide data. One of the variants enhances the representational capacity of the data using so-called feature line segments, while all three consider the concept of a territorial hypersphere per prototype feature point. Additionally, this experimental comparison is entirely reproducible, as Python implementations are provided for all the methods and the main simulation, and the experiments are performed using three publicly available datasets: two related to slope stability and one for landslide susceptibility. Results show that the three variants are very competitive and easily applicable.

Square Static – Deep Hyper Optimization and Genetic Meta-Learning Approach for Disease Classification

A recent study showed that machine learning algorithms are trained to diagnose the disease in accurate manner. The stroke prediction of people based on various features is identified effectively using machine learning models. The main objective of this work is to use an optimized model for breast cancer and stroke prediction in patients. Two different datasets are collected and their results were obtained. In the first phase, the square static method is applied to find the correlation among the features and this enhances the dimensionality reduction in the dataset. In the second phase, Deep Hyper Optimization and Genetic Meta-Learning Model (DHO-GML) are applied to perform the classification efficiently by selecting the optimized model. The performance of the proposed work is compared with the models such as Logistic Regression (LR), XGBClassifier, Support Vector Machine (SVM), K Nearest Neighbors (KNN), Bernouli NB and Ada Boost classifier. As shown in the analysis the proposed model produces an accuracy above 99% with the metrics precision, recall, F1 score, Area under Curve (AUC) and elapsed time.

Prototype generation method using a growing self-organizing map applied to the banking sector

In fields like security risk analysis, Fast Moving Consumer Goods, Internet of Things, or the banking sector, it is necessary to deal with large datasets containing a great list of variables. In these situations, the analysis becomes intricate and computationally expensive, so data reduction techniques play an important role. Prototype generation methods provide a reduced dataset with the same properties as the original. GSOMs (growing self-organizing maps) reduce the data size without the need for prefixing the number of neurons needed to represent the input space. To the best of the authors’ knowledge, this is the first time that the GSOM is applied for reduction and generation of prototypes, posing an advantage over their predecessors, the SOMs (self-organizing maps), which do not have the automatic growth feature. This work addresses the use of a GSOM to reduce the number of prototypes to use in a 1-NN (1 nearest neighbor) classifier. The proposed methodology is applied to an income dataset for testing and a large bank dataset that contain classifications into two different groups. The 1-NN classifier is used to obtain predictions using the nodes of the GSOM as prototypes. This article demonstrates that GSOMs save a significant amount of time in obtaining nearly the same validation results as SOMs by comparing the classifications obtained in the bank dataset. The results show data reductions of more than 99%, and accuracies greater than 80% for the income dataset and 74% for the bank dataset.

Could Gut Microbiota Composition Be a Useful Indicator of a Long-Term Dietary Pattern?

Despite the known effects of diet on gut microbiota composition, not many studies have evaluated the relationship between distinct dietary patterns and gut microbiota. The aim of our study was to determine whether gut microbiota composition could be a useful indicator of a long-term dietary pattern. We collected data from 89 subjects adhering to omnivorous, vegetarian, vegan, and low-carbohydrate, high-fat diet that were equally distributed between groups and homogenous by age, gender, and BMI. Gut microbiota composition was analyzed with a metabarcoding approach using V4 hypervariable region of the 16S rRNA gene. K-means clustering of gut microbiota at the genus level was performed and the nearest neighbor classifier was applied to predict microbiota clustering classes. Our results suggest that gut microbiota composition at the genus level is not a useful indicator of a subject’s dietary pattern, with the exception of a vegan diet that is represented by a high abundance of Prevotella 9. Based on our model, a combination of 26 variables (anthropometric measurements, serum biomarkers, lifestyle factors, gastrointestinal symptoms, psychological factors, specific nutrients intake) is more important to predict an individual’s microbiota composition cluster, with 91% accuracy, than the dietary intake alone. Our findings could serve to develop strategies to educate individuals about changes of some modifiable lifestyle factors, aiming to classify them into clusters with favorable health markers, independent of their dietary pattern.

ExHiF: Alzheimer's disease detection using exemplar histogram-based features with CT and MR images

The classification of medical images is an important priority for clinical research and helps to improve the diagnosis of various disorders. This work aims to classify the neuroradiological features of patients with Alzheimer's disease (AD) using an automatic hand-modeled method with high accuracy. This work uses two (private and public) datasets. The private dataset consists of 3807 magnetic resonance imaging (MRI) and computer tomography (CT) images belonging to two (normal and AD) classes. The second public (Kaggle AD) dataset contains 6400 MR images. The presented classification model comprises three fundamental phases: feature extraction using an exemplar hybrid feature extractor, neighborhood component analysis-based feature selection, and classification utilizing eight different classifiers. The novelty of this model is feature extraction. Vision transformers inspire this phase, and hence 16 exemplars are generated. Histogram-oriented gradients (HOG), local binary pattern (LBP) and local phase quantization (LPQ) feature extraction functions have been applied to each exemplar/patch and raw brain image. Finally, the created features are merged, and the best features are selected using neighborhood component analysis (NCA). These features are fed to eight classifiers to obtain highest classification performance using our proposed method. The presented image classification model uses exemplar histogram-based features; hence, it is called ExHiF. We have developed the ExHiF model with a ten-fold cross-validation strategy using two (private and public) datasets with shallow classifiers. We have obtained 100% classification accuracy using cubic support vector machine (CSVM) and fine k nearest neighbor (FkNN) classifiers for both datasets. Our developed model is ready to be validated with more datasets and has the potential to be employed in mental hospitals to assist neurologists in confirming their manual screening of AD using MRI/CT images.

Multi-featured multi-scale combination of high-resolution remote sensing images for building extraction

Abstract As one of the important feature categories in urban geographic data, buildings are the key thematic elements to be represented in large-scale urban mapping with the high speed of urban digital construction. The identification and extraction of buildings are of great significance for feature extraction, feature matching, image interpretation and mapping. However, the great variability of building size, shape, color, orientation, etc., in remote sensing images poses a great challenge to building detection. To this end, this paper proposes an algorithm based on multi-feature multi-scale fusion for the automatic extraction of buildings in remote sensing images are represented in the form of roofs. It is difficult to represent all buildings with a single feature because of the different colors, textures and shapes of building roofs. Effective features to describe buildings are proposed, including edge density and edge distribution, brightness contrast, color contrast and other features to describe building edge brightness. We propose effective features to describe buildings, including edge density and edge distribution, luminance contrast, color contrast and other underlying features to describe the edges, luminance and color of buildings, and adding special structural features such as main direction orthogonality and target integrity and symmetry to describe buildings by multiple features together. Moreover, the K-value nearest neighbor classification algorithm is used to train a series of samples, and the weights of each feature in the multi-feature model are obtained through iterative learning to obtain the multi-feature linear model and calculate the visual saliency of buildings in the sliding window; finally, the proposed algorithm has experimented with several groups of high-resolution remote sensing images respectively, and the multi-scale multi-feature fusion model algorithm is used as the Erkoff random field model to compare the algorithm. The results of this paper show that the proposed multiscale multi-feature fusion model algorithm improves by 10.82% for building classification accuracy extraction and 13.96% for feature selection extraction accuracy, and finally, the comparison from the shape optimization effect figure concludes that the multiscale multi-feature fusion model can achieve better extraction accuracy and practical effect for buildings in remote sensing images, which has certain practicality and It has certain practicality and superiority. It promotes the in-depth application of multi-feature multi-scale combined high-resolution remote sensing image-building extraction in geographic states, road traffic and other industries.t

Universal consistency of Wasserstein k-NN classifier: a negative and some positive results

Abstract We study the $k$-nearest neighbour classifier ($k$-NN) of probability measures under the Wasserstein distance. We show that the $k$-NN classifier is not universally consistent on the space of measures supported in $(0,1)$. As any Euclidean ball contains a copy of $(0,1)$, one should not expect to obtain universal consistency without some restriction on the base metric space, or the Wasserstein space itself. To this end, via the notion of $\sigma $-finite metric dimension, we show that the $k$-NN classifier is universally consistent on spaces of discrete measures (and more generally, $\sigma $-finite uniformly discrete measures) with rational mass. In addition, by studying the geodesic structures of the Wasserstein spaces for $p=1$ and $p=2$, we show that the $k$-NN classifier is universally consistent on spaces of measures supported on a finite set, the space of Gaussian measures and spaces of measures with finite wavelet series densities.

Comparison of performances of ML-Algorithms in the estimation of the execution time of non-parallel Java programs

Performance enhancement of a computer program is an important aspect of today's world. Developers produce programs, and there is a lack of accurate methods for predicting the execution time of a computer program prior to its execution in an executable environment. Predicting the execution time of a particular program before execution would be great for developing the program with the highest performance efficiency and with the lowest execution latency. This research introduces a Machine Learning based solution to predict an execution-time-based label for a given computer program. There are three main types of parameters in a computer program that affect the execution time, such as Static Code Features, HTTP Calls, and the Hardware Performance of the execution environment. In this research, the Machine Learning model was trained for the parameters of the above types (Programs with Static Programs & HTTP calls) by executing them on a fixed hardware infrastructure execution condition. We analyzed the number of if conditions, methods, breaks, switches, loops, nested-loop-depth, frequencies, and the behaviour of HTTP calls, kind of features of a computer program in order to generate an accurate execution time complexity prediction label of a computer program. Further, in the collected data set, the most prominent feature which affects the complexity among the features that we considered is the number of HTTP calls and nested loop depth, followed by loops. Accuracy Score, Precision, Recall, and F1 Score values of the ML model were generated for the traditional classification algorithms such as Decision Tree Classifier, K Nearest Neighbor Classifier, Random Forest Classifier, Naive Bayes Classifier, Support Vector Classifier, and MLP Classifiers in order to verify the effectiveness of the model. The best accuracy score was achieved with an overall 88% by using the approach of Random Forest. The findings of this research can be optimized for implementing an IDE plugin or a developer tool that can predict the exact execution time of a given computer program live by integrating the specifications of the execution device. It will help developers in terms of optimizing a particular computer program and developing it for a minimum execution latency and enhancing the performance of the program.

Parameterizing the cost function of dynamic time warping with application to time series classification

Dynamic time warping (DTW) is a popular time series distance measure that aligns the points in two series with one another. These alignments support warping of the time dimension to allow for processes that unfold at differing rates. The distance is the minimum sum of costs of the resulting alignments over any allowable warping of the time dimension. The cost of an alignment of two points is a function of the difference in the values of those points. The original cost function was the absolute value of this difference. Other cost functions have been proposed. A popular alternative is the square of the difference. However, to our knowledge, this is the first investigation of both the relative impacts of using different cost functions and the potential to tune cost functions to different time series classification tasks. We do so in this paper by using a tunable cost function lambda _{gamma } with parameter gamma . We show that higher values of gamma place greater weight on larger pairwise differences, while lower values place greater weight on smaller pairwise differences. We demonstrate that training gamma significantly improves the accuracy of both the { DTW } nearest neighbor and Proximity Forest classifiers.

Radio Frequency Fingerprint Identification Based on Metric Learning

With the popularization of the internet of things (IoT), its security has become increasingly prominent. Radio-frequency fingerprinting (RFF) is used as a physical-layer security method to provide security in wireless networks. However, the problems of poor performance in a highly noisy environment and less consideration of calculation resources are urgent to be resolved in a practical RFF application domain. The authors propose a new RFF identification method based on metric learning. They used power spectrum density (PSD) to extract the RFF from the nonlinearity of the RF front end. Then they adopted the large margin nearest neighbor (LMNN) classification algorithm to identify eight software-defined radio (SDR) devices. Different from existing RFF identification algorithms, the proposed LMNN method is more general and can learn the optimal metric from the wireless communication environment. Furthermore, they propose a new training and test strategy based on mixed SNR, which significantly improves the performance of conventional low-complexity RFF identification methods. Experimental results show that the proposed method can achieve 99.8% identification accuracy with 30dB SNR and 96.83% with 10dB SNR. In conclusion, the study demonstrates the effectiveness of the proposed method in recognition efficiency and computational complexity.

Predicting Intrusion in a Network Traffic Using Variance of Neighboring Object’s Distance

Activities in network traffic can be broadly classified into two categories: normal and malicious. Malicious activities are harmful and their detection is necessary for security reasons. The intrusion detection process monitors network traffic to identify malicious activities in the system. Any algorithm that divides objects into two categories, such as good or bad, is a binary class predictor or binary classifier. In this paper, we utilized the Nearest Neighbor Distance Variance (NNDV) classifier for the prediction of intrusion. NNDV is a binary class predictor and uses the concept of variance on the distance between objects. We used KDD CUP 99 dataset to evaluate the NNDV and compared the predictive accuracy of NNDV with that of the KNN or K Nearest Neighbor classifier. KNN is an efficient general purpose classifier, but we only considered its binary aspect. The results are quite satisfactory to show that NNDV is comparable to KNN. Many times, the performance of NNDV is better than KNN. We experimented with normalized and unnormalized data for NNDV and found that the accuracy results are generally better for normalized data. We also compared the accuracy results of different cross validation techniques such as 2 fold, 5 fold, 10 fold, and leave one out on the NNDV for the KDD CUP 99 dataset. Cross validation results can be helpful in determining the parameters of the algorithm.

A dataset of ecological factors in the distribution area of Tianshan spruce in Yili region in 2014

Tianshan spruce (Picea schrenkiana var. tianschanica ) is an important tree for water conservation, windbreak and sand fixation in the mountains of Xinjiang, and plays an important role in the formation and maintenance of forest ecosystem in Xinjiang. Ecological factors such as temperature, precipitation, soil and topography are important data bases for studying forest ecosystem. Based on multi-source data such as remote sensing image, DEM, meteorological data and soil data in Yili region of Xinjiang, we used multi-scale segmentation, nearest neighbor classification and spatial analysis to generate a dataset of ecological factors in the distribution area of Tianshan spruce in Yili region in 2014, covering six ecological factors, namely temperature, precipitation, sunshine duration, slope, slope direction and soil type. In this dataset, strict procedures of data quality control were carried out by TTA Mask accuracy verification and other methods to ensure the accuracy and reliability of the data. The dataset can provide data support for forest ecosystem health assessment.