Nearest-neighbor Classifier Research Articles

Sensor Selection Embedded in Active Fault Diagnosis Algorithms

A methodology is presented for simultaneous sensor selection and design of fault diagnosis tests in complex systems, for which steady-state or dynamic models are available. The method assesses all possible sensor combinations for their information with respect to system faults in the presence of uncertainty and chooses sensors based on their contribution to information gain. In particular, sensors are cast as binary variables in the normalized Fisher information matrix, which is used to calculate the optimal fault detection and isolation (FDI) test designs by manipulating admissible system inputs. Test design optimization is always naturally embedded in the proposed method. Then, the Kullback–Leibler divergence and the Hellinger distance are used to explore the isolation capability of an FDI test when uncertainty is considered for the system inputs and parameters. The FDI tests are deployed and verified using a principal component analysis and the $k$ -nearest neighbor classification algorithm. The capability of the method to detect and isolate faults at high correct classification rates and low false alarm rates is demonstrated. The tool-chain proposed is tested on a benchmark three-tank system at various levels of measurement noise and uncertainty. The benefits and drawbacks of each step of the proposed method are assessed and discussed along with their computational footprint.

Efficient video face recognition based on frame selection and quality assessment.

The article is considering the problem of increasing the performance and accuracy of video face identification. We examine the selection of the several best video frames using various techniques for assessing the quality of images. In contrast to traditional methods with estimation of image brightness/contrast, we propose to utilize the deep learning techniques that estimate the frame quality by using the lightweight convolutional neural network. In order to increase the effectiveness of the frame quality assessment step, we propose to distill knowledge of the cumbersome existing FaceQNet model for which there is no publicly available training dataset. The selected K-best frames are used to describe an input set of frames with a single average descriptor suitable for the nearest neighbor classifier. The proposed algorithm is compared with the traditional face feature extraction for each frame, as well as with the known clustering methods for a set of video frames.

Extraction of Multiple Diseases in Apple Leaf Using Machine Learning

This paper proposes a novel algorithm of segmentation of diseased part in apple leaf images. In agriculture-based image processing, leaf diseases segmentation is the main processing task for region of interest extraction. It is also extremely important to segment the plant leaf from the background in case on live images. Automated segmentation of plant leaves from the background is a common challenge in the processing of plant images. Although numerous methods have been proposed, still it is tough to segment the diseased part of the leaf from the live leaf images accurately by one particular method. In the proposed work, leaves of apple having different background have been segmented. Firstly, the leaves have been enhanced by using Brightness-Preserving Dynamic Fuzzy Histogram Equalization technique and then the extraction of diseased apple leaf part is done using a novel extraction algorithm. Real-time plant leaf database is used to validate the proposed approach. The results of the proposed novel methodology give better results when compared to existing segmentation algorithms. From the segmented apple leaves, color and texture features are extracted which are further classified as marsonina coronaria or apple scab using different machine learning classifiers. Best accuracy of 96.4% is achieved using K nearest neighbor classifier.

Modeling vehicle ownership with machine learning techniques in the Greater Tamale Area, Ghana.

Vehicle ownership modeling and prediction is a crucial task in the transportation planning processes which, traditionally, uses statistical models in the modeling process. However, with the advancement in computing power of computers and Artificial Intelligence, Machine Learning (ML) algorithms are becoming an alternative or a complement to the statistical models in modeling the transportation planning processes. Although the application of ML algorithms to the transportation planning processes—like mode choice, and traffic forecasting and demand modeling—have received much attention in research and abound in literature, scanty attention is paid to its application to vehicle ownership modeling especially in the context of small to medium cities in developing countries. Therefore, this study attempts to fill this gap by modeling vehicle ownership in the Greater Tamale Area (GTA), a typically small to medium city in Ghana. Using a cross sectional survey of formal sectors workers, data was collected between June–August 2018. The study applied nine different ML classification algorithms to the dataset using 10-fold cross-validation technique/s and the Cohen-Kappa static/statistic to evaluate the predictive performance of each of the algorithms, and the Permutation Feature Importance to examine the features that contribute significantly to the prediction of vehicle ownership in GTA. The results showed that Linear Support Vector Classification (LinearSVC) classifier performed well in comparison with the other classifiers with regards to the overall predictive ability of the classifiers. In terms of class predictions, K- Nearest Neighbors (KNN) classifier performs well for no-vehicle class whiles Linear Support Vector Classification (LinearSVC) and GaussianNB classifiers performs well for motorcycle ownership. LinearSVC and Logistic Regression classifiers performed well on the car ownership class. Also, the results indicated that travel mode choice, average monthly income, average travel distance to workplace, average monthly expenditure on transport, duration of travel to workplace, occupational rank, age, household size and marital status were significant in predicting vehicle ownership for most of the classifiers. These findings could help policies makers carve out strategies that would reduce vehicle ownership but improve personal mobility.

Application of Petersen graph pattern technique for automated detection of heart valve diseases with PCG signals

This work aimed to use machine learning to diagnose four heart valve disease conditions and normal heart sounds. This paper proposed the automated classification of normal, aortic stenosis, mitral valve prolapse, mitral stenosis, and mitral regurgitation using phonocardiogram (PCG) signals.This work proposed a novel graph-based feature generator developed using a graph-based technique called Petersen graph pattern (PGP). In addition, a new decomposition model was proposed using variable-sized overlapping blocks, namely tent pooling (TEP) decomposition. By combining TEP and PGP, a novel multilevel feature generation network was developed. Iterative neighborhood component analysis (INCA) was used to select the features. The selected features were fed to decision tree (DT), linear discriminant (LD), bagged tree (BT), and support vector machine (SVM) classifiers for automated classification into five classes.The proposed method's results yielded 100.0% classification accuracy using the k nearest neighbor (kNN) classifier with a ten-fold cross-validation strategy in classifying the five classes. DT, LD, BT, SVM classifiers yielded an accuracy of 95.10%, 98.30%, 98.60%, and 99.90%, respectively.Our attained high classification accuracy suggests that the proposed PGP and TEP based model can be used for heart sound classification using PCG signals.

Use of artificial intelligence on Electroencephalogram (EEG) waveforms to predict failure in early school grades in children from a rural cohort in Pakistan.

Universal primary education is critical for individual academic growth and overall adult productivity of nations. Estimates indicate that 25% of 59 million primary age out of school children drop out and early grade failure is one of the factors. An objective and feasible screening measure to identify at-risk children in the early grades can help to design appropriate interventions. The objective of this study was to use a Machine Learning algorithm to evaluate the power of Electroencephalogram (EEG) data collected at age 4 in predicting academic achievement at age 8 among rural children in Pakistan. Demographic and EEG data from 96 children of a cohort along with their academic achievement in grade 1–2 measured using an academic achievement test of Math and language at the age of 7–8 years was used to develop the machine learning algorithm. K- Nearest Neighbor (KNN) classifier was used on different model combinations of EEG, sociodemographic and home environment variables. KNN model was evaluated using 5 Stratified Folds based on the sensitivity and specificity. In the current dataset, 55% and 74% failed in the mathematics and language test respectively. On testing data across each fold, the mean sensitivity and specificity was calculated. Sensitivity was similar when EEG variables were combined with sociodemographic, and home environment (Math = 58.7%, Language = 66.3%) variables but specificity improved (Math = 43.4% to 50.6% and Language = 32% to 60%). The model requires further validation for EEG to be used as a screening measure with adequate sensitivity and specificity to identify children in their preschool age who may be at high risk of failure in early grades.

Nearest neighbour methods and their applications in design of 5G & beyond wireless networks

In this paper, we present an overview of Nearest neighbour (NN) methods, which are frequently employed for solving classification problems using supervised learning. The article concisely introduces the theoretical background, algorithmic, and implementation aspects along with the key applications. From an application standpoint, this article explores the challenges related to the 5G and beyond wireless networks which can be solved using NN classification techniques.

Analysis and Prediction Of Pima Indian Diabetes Dataset Using SDKNN Classifier Technique

The newly proposed weighted k nearest neighbour is known as standard deviation K nearest neighbour(SDKNN) classifier technique. It is based on the principle of standard deviation. Standard deviation measures spreading of attribute about mean. Spreading of attribute plays a significant role to improve the classification accuracy of a dataset. Most of our distance calculation method between two points is determined by using euclidean distance process for finding nearest neighbour. Our proposed technique is based on a new distance calculation formula to find nearest neighbour in KNN. We apply here standard deviations of attributes as power for calculating distance between train dataset and test dataset. Distance calculation between two points in k nearest neighbour classifier is modified according to the standard deviation of attribute. In this paper, standard deviation of attributes are used. In first attempt, we have used standard deviation of attributes as power for calculating K Nearest Neighbour to improve classification accuracy and in second attempt, based on mean of standard deviation attributes, distance in K Nearest Neighbour is processed to further improve the classification accuracy. Our concept is implemented on Pima Indian Diabetes Dataset (PIDD). The analysis on Pima Indian Diabetes Dataset (PIDD) is carried out by splitting dataset in to 90% training data and 10% testing data. We have found that, in our proposed technique, average classification accuracy gives result 83.2%, a great improvement as compared to other conventional technique.

An Improved K Nearest Neighbor Classifier for High-Dimensional and Mixture Data

Conventional K nearest neighbor classifier is challenged by data with high-dimensional and mixture of continuous and categorical variable characteristics. To address this issue, we propose an improved K nearest neighbor classifier by using principal component mixed also called PCAmix. First, we employed the PCAmix to preprocess high-dimensional and mixture data. Then, the KNN classifier is used to classify objects characterized by high-dimensional mixture of continuous and categorical variables. We have evaluated our method using five mixture datasets that are available from the UCI machine learning repository and compared our results with the base line approaches. The experimental results showed that the proposed method had better classification performance for high-dimensional mixture data.

A genetic algorithm approach for predicting ribonucleic acid sequencing data classification using KNN and decision tree

Malaria larvae accept explosive variable lifecycle as they spread across numerous mosquito vector stratosphere. Transcriptomes arise in thousands of diverse parasites. Ribonucleic acid sequencing (RNA-seq) is a prevalent gene expression that has led to enhanced understanding of genetic queries. RNA-seq tests transcript of gene expression, and provides methodological enhancements to machine learning procedures. Researchers have proposed several methods in evaluating and learning biological data. Genetic algorithm (GA) as a feature selection process is used in this study to fetch relevant information from the RNA-Seq Mosquito Anopheles gambiae malaria vector dataset, and evaluates the results using kth nearest neighbor (KNN) and decision tree classification algorithms. The experimental results obtained a classification accuracy of 88.3 and 98.3 percents respectively.

Comparison of Different Dimension Reduction Methods in Classification of Hyperspectral Images

In remote sensing, which is becoming increasingly important today, researchers use high-dimensional data representing the surface of the earth to find relationships between various spectral signatures. In particular, images can consist of hundreds of high-resolution bands that reflect the properties of different materials. However, the presence of a large number of different bands in high-dimensional space can make interpretation of these features difficult. Various difficulties are encountered due to dimensionality problem for pre-processing of remote sensing data. Research in this area reveals that this is a difficult problem and not a single solution to all problems. However, recent studies show that manifold learning techniques are a very important solution in the preprocessing of hyperspectral images. In this study, the performance of the state-of-the-art manifold embedding methods on hyperspectral data is analyzed comparatively. The dimension reduction application of each method has been carried out by using two different data sets that are used most in this field and their performance have been verified by the nearest neighbor (1NN) classification. Even though there are class-based differences in the classification of hyperspectral data, it is seen that manifold embedding methods, which are compared according to the obtained results, yield successful results. In addition, the runtime of each method is presented graphically and explained along with the reasons for which method works faster.

Klasifikasi Text Judul Buku Perpustakaan Untuk Menentukan Kategori Buku Menggunakan K-Nearest Neighbor

The need for information in the form of books or scientific articles at the Library of UIN Raden Fatah Palembang continues to increase. To make it easier to find book information, one of which is by classifying books based on the type of category. In classifying library book data, the Nearest Neighbor Classifier method in data mining can be combined with text data extraction techniques to classify library book title text data. The purpose of this study was to classify the text title of library books using the Nearest Neighbor Classifier to determine the type of book category. This research method uses the Nearest Neighbor Classifier data mining classification technique. The results of this study are that the highest accuracy value is found at K = 12, which is 72.50%, and the model formed can be used to classify books with labels 2x0, 150, 2x2, 400, 020, 2x1, 657, 500, 375, 302.2, 800. and cannot be used for classifying books with class labels 070, 370, 330, 300, 600, 340, 700.

A long-term record (1995–2019) of the dynamics of land desertification in the middle reaches of Yarlung Zangbo River basin derived from Landsat data

Widespread desertification in the middle part of the Yarlung Zangbo River (YZR) basin is threatening the sustainable development of this region. To capture this process, a method was proposed for large-scale desertification monitoring by using Landsat images from 1995 to 2019. The method used an integrated classification method combined with a hierarchical decision tree and nearest neighbor classifiers. The spatio-temporal dynamics of the desertification pattern were analyzed to assist in the detection of possible driving forces. Using validation samples collected from Google Earth high-resolution images and field investigations, the overall accuracy of the classification in 2019 was 92.3% with a Kappa coefficient of 0.84. The major results were: (1) total sandy land area in 2019 was 734.1 km2, which accounted for 3.7% of the study area, prominently distributed along the wide river valleys and inlets of tributaries with a strip and discontinuous pattern. Sandy land tends to be distributed in the southern aspect regions with lower elevations and that are closer to rivers; (2) sandy land areas showed two temporal stages: a gradual increase of 102.4 km2 from 1995 to 2015 and a large decrease of 106.8 km2 from 2015 to 2019; (3) newly increased sandy land was distributed in the YZR Valley, while the revegetation on sandy land occurred mainly in the Lhasa River basin and some regions in the YZR Valley; and (4) increased sandy land area of 142.1 km2 was mainly distributed in the southern band of the two rivers. Correspondingly, revegetation on sandy land was more effective on the northern banks of the river valleys. These findings provide guidance for implementing vegetation recovery on sandy lands and provide important insights for maintaining sustainable development.

A new fractal H-tree pattern based gun model identification method using gunshot audios

BackgroundGun model identification (GMI) is a complex issue for digital forensics examiners/professions. Because the GMI process is a highly costed process, and it is generally detected manually. A sound classification model is presented in this research to decrease the cost of the GMI and automate this process. Material and methodThe primary objective of this research is to present a new intelligent audio forensics tool. Therefore, a new gunshot dataset was collected, and the collected dataset includes 2130 audios of the 28 gun models. This dataset can be downloaded using http://web.firat.edu.tr/sdogan/Gun_S_Dogan.rar link. The presented fractal H-tree pattern-based classification method is applied to these audios to obtain results. This method has three fundamental phases, and these are feature extraction, the most informative features selection, and classification. This method uses both a fractal textural generator and statistical features. By deploying tunable q-factor wavelet transform (TQWT), a multileveled feature generation method is created to generate both low-level and high-level features. The recommended fractal H-tree pattern and statistical feature extraction functions generate features at each level. Neighborhood component analysis (NCA) chooses the most informative features. In the classification phase, the support vector machine (SVM) and k nearest neighbor (kNN) classifiers are used. ResultsThe recommended fractal H-tree pattern-based method yielded 96.10% and 90.40% by employing kNN and SVM, respectively. ConclusionThe calculated results and findings denoted the high classification capability of the presented fractal H-tree pattern-based method for gun model classification using gunshot audios. Also, this research shows that a new audio forensic tool can be developed by employing the presented method for GMI.

Improving the Accuracy of Nearest-Neighbor Classification Using Principled Construction and Stochastic Sampling of Training-Set Centroids.

A conceptually simple way to classify images is to directly compare test-set data and training-set data. The accuracy of this approach is limited by the method of comparison used, and by the extent to which the training-set data cover configuration space. Here we show that this coverage can be substantially increased using coarse-graining (replacing groups of images by their centroids) and stochastic sampling (using distinct sets of centroids in combination). We use the MNIST and Fashion-MNIST data sets to show that a principled coarse-graining algorithm can convert training images into fewer image centroids without loss of accuracy of classification of test-set images by nearest-neighbor classification. Distinct batches of centroids can be used in combination as a means of stochastically sampling configuration space, and can classify test-set data more accurately than can the unaltered training set. On the MNIST and Fashion-MNIST data sets this approach converts nearest-neighbor classification from a mid-ranking- to an upper-ranking member of the set of classical machine-learning techniques.

Minimax Rate Optimal Adaptive Nearest Neighbor Classification and Regression

k Nearest Neighbor (kNN) method is a simple and popular statistical method for classification and regression. For both classification and regression problems, existing works have shown that, if the distribution of the feature vector has bounded support and the probability density function is bounded away from zero in its support, the convergence rate of the standard kNN method, in which k is the same for all test samples, is minimax optimal. On the contrary, if the distribution has unbounded support, we show that there is a gap between the convergence rate achieved by the standard kNN method and the minimax bound. To close this gap, we propose an adaptive kNN method, in which different k is selected for different samples. Our selection rule does not require precise knowledge of the underlying distribution of features. The proposed adaptive method significantly outperforms the standard one. We characterize the convergence rate of the proposed adaptive method, and show that it matches the minimax lower bound.

Acceleration of Large Margin Metric Learning for Nearest Neighbor Classification Using Triplet Mining and Stratified Sampling

Metric learning is a technique in manifold learning to find a projection subspace for increasing and decreasing the inter- and intra-class variances, respectively. Some metric learning methods are based on triplet learning with anchor-positive-negative triplets. Large margin metric learning for nearest neighbor classification is one of the fundamental methods to do this. Recently, Siamese networks have been introduced with the triplet loss. Many triplet mining methods have been developed for Siamese nets; however, these techniques have not been applied on the triplets of large margin metric learning. In this work, inspired by the mining methods for Siamese nets, we propose several triplet mining techniques for large margin metric learning. Moreover, a hierarchical approach is proposed, for acceleration and scalability of optimization, where triplets are selected by stratified sampling in hierarchical hyper-spheres. We analyze the proposed methods on three publicly available datasets.

A cooperative coevolution framework for evolutionary learning and instance selection

Data science becomes an emerging topic in developing computational and artificial intelligence techniques due to the rapid growth of information and data over past decade. Data-driven evaluation, through scanning data to validate the performance of a method, is the crux of effectively retrieving information. Ordinarily, the computational cost at data-driven evaluation is proportional to the volume, determined by the number of instances and the number of attributes, of the data. One straightforward way of reducing the volume of data is instance selection. Past studies have adopted evolutionary algorithm (EA) for instance selection, yet these studies are mainly based on k nearest neighbors classifier due to the high computational cost at learning process. Leveraging EA to learn patterns, models, or classifiers forms a famous branch of learning method known as evolutionary learning (EL). Nevertheless, EL requires immense amount of data-driven fitness evaluations, which detracts EL from efficiency and causes slow convergence. The issue of high computational cost at data-driven evaluation restricts the capability of evolutionary learning and evolutionary instance selection. This study proposes a novel framework called cooperative evolutionary learning and instance selection (CELIS) for addressing the above issues. There are three main features in the proposed CELIS: 1) cooperative evolutionary learning and instance selection in an adaptive manner, 2) fast solution evaluation based on the selected subset of data for evolutionary learning, and 3) effective data evaluation of representativeness as per promising solution for evolutionary instance selection. This study conducts a series of experiments for testing the effectiveness and efficiency of the proposed CELIS on data mining, classification, and symbolic regression problems. The results show that the proposed method can significantly enhance the performance of state-of-the-art EAs on the three types of problems in terms of solution quality and convergence speed. Further analysis also validates that CELIS is able to gain a small and representative subset of data.

A secure distributed machine learning protocol against static semi-honest adversaries

Machine learning has been successfully applied to various fields over the last few years. However, it still faces two critical challenges. On one hand, the concern for the security issue in machine learning is increased. On the other hand, data exists in the form of isolated islands across different organizations. In this work, we focus on the privacy-preserving issue on a non-parametric machine learning algorithmize, i.e., the k Nearest Neighbor Classification (k NNC) in which, training data are split vertically among multiple servers. We propose a novel protocol that is secure against static semi-honest adversaries. In specific, the clients can obtain the label of his/her query without disclosing the servers’ data, the client’s query, and the client’s output to others. We use the state-of-the-art lattice-based fully homomorphic encryption to realize the privacy-preserving distance computation. In order to protect data access patterns, permutation technique and Oblivious Transfer are used in the top-k selection phase. We proved the security via the simulation paradigm. Meanwhile, we implemented our protocol and performed extensive experiments. Results show that our protocol performs well, especially in a large-width environment. Compared to the existing solution, our protocol leaks no information about the participants’ private input and output in both centralized and distributed architectures. Meanwhile, our protocol runs faster than existing solutions.

Challenges in KNN Classification

The KNN algorithm is one of the most popular data mining algorithms. It has been widely and successfully applied to data analysis applications across a variety of research topics in computer science. This paper illustrates that, despite its success, there remain many challenges in KNN classification, including K computation, nearest neighbor selection, nearest neighbor search and classification rules. Having established these issues, recent approaches to their resolution are examined in more detail, thereby providing a potential roadmap for ongoing KNN-related research, as well as some new classification rules regarding how to tackle the issue of training sample imbalance. To evaluate the proposed approaches, some experiments were conducted with 15 UCI benchmark datasets.