K-nearest Neighbor Classifier Research Articles

RepEKShot: an evidential k-nearest neighbor classifier with repulsion loss for few-shot named entity recognition

RepEKShot: an evidential k-nearest neighbor classifier with repulsion loss for few-shot named entity recognition

Quantum K-Nearest Neighbors: Utilizing QRAM and SWAP-Test Techniques for Enhanced Performance

This work introduces a quantum K-Nearest Neighbor (K-NN) classifier algorithm. The algorithm utilizes angle encoding through a Quantum Random Access Memory (QRAM) using n number of qubit addresses with O(log(n)) space complexity. It incorporates Grover’s algorithm and the quantum SWAP-Test to identify similar states and determine the nearest neighbors with high probability, achieving Om search complexity, where m is the qubit address. We implement a simulation of the algorithm using IBM’s Qiskit with GPU support, applying it to the Iris and MNIST datasets with two different angle encodings. The experiments employ multiple QRAM cell sizes (8, 16, 32, 64, 128) and perform ten trials per size. According to the performance, accuracy values in the Iris dataset range from 89.3 ± 5.78% to 94.0 ± 1.56%. The MNIST dataset’s mean binary accuracy values range from 79.45 ± 18.84% to 94.00 ± 2.11% for classes 0 and 1. Additionally, a comparison of the results of this proposed approach with different state-of-the-art versions of QK-NN and the classical K-NN using Scikit-learn. This method achieves a 96.4 ± 2.22% accuracy in the Iris dataset. Finally, this proposal contributes an experimental result to the state of the art for the MNIST dataset, achieving an accuracy of 96.55 ± 2.00%. This work presents a new implementation proposal for QK-NN and conducts multiple experiments that yield more robust results than previous implementations. Although our average performance approaches still need to surpass the classic results, an experimental increase in the size of QRAM or the amount of data to encode is not achieved due to limitations. However, our results show promising improvement when considering working with more feature numbers and accommodating more data in the QRAM.

A simplified method for relapsing-remitting multiple sclerosis detection: Insights from resting EEG signals

Background and Objective:Multiple sclerosis (MS) is a neurodegenerative autoimmune disease affecting the central nervous system, leading to various neurological symptoms. Early detection is paramount to prevent enduring damage during MS episodes. Although magnetic resonance imaging (MRI) is a common diagnostic tool, this study aims to explore the feasibility of using electroencephalography (EEG) signals for MS detection, considering their accessibility and ease of application compared to MRI. Methods:The study involved the analysis of EEG signals during rest from 17 MS patients and 27 healthy volunteers to investigate MS-healthy patterns. Power spectral density features (PSD) were extracted from the 32-channel EEG signals. The study employed Linear Discriminant Analysis (LDA), Support Vector Machine (SVM), Classification and Regression Trees (CART), and k-Nearest Neighbor (kNN) classifiers to identify channels with the highest accuracy. Notably, the study achieved 100% accuracy in MS detection using the “Fp1” and “Pz” channels with the LDA classifier. A statistical analysis, utilizing the independent sample t-test, was conducted to explore whether PSD features of these channels differed significantly between healthy individuals and those with MS. Results:The results of the study demonstrate that effective detection of MS can be achieved using PSD features from only two channels of the EEG signal. Specifically, the “Fp1” and “Pz” channels exhibited 100% accuracy in MS detection with the LDA classifier. The statistical analysis further explored and confirmed the significant differences in PSD features between healthy individuals and MS patients. Conclusion:The study concludes that the proposed method, utilizing PSD features from specific EEG channels, offers a straightforward and efficient diagnostic approach for the effective detection of MS. The findings suggest the potential utility of EEG signals as a non-invasive and accessible alternative for MS detection, highlighting the importance of further research in this direction.

Newly identified Phonocardiography frequency bands for psychological stress detection with Deep Wavelet Scattering Network

The timely psychological stress detection can improve the quality of human life by preventing stress-induced behavioral and pathological consequences. This paper presents a novel framework that eliminates the need of Electrocardiography (ECG) signals-based referencing of Phonocardiography (PCG) signals for psychological stress detection. This stand-alone PCG-based methodology uses wavelet scattering approach on the data acquired from twenty-eight healthy adult male and female subjects to detect psychological stress. The acquired PCG signals are asynchronously segmented for the analysis using wavelet scattering transform. After the noise bands removal, the optimized segmentation length (L), scattering network parameters namely-invariance scale (J) and quality factor (Q) are utilized for computation of scattering features. These scattering coefficients generated are fed to K-nearest neighbor (KNN) and Extreme Gradient Boosting (XGBoost) classifier and the ten-fold cross validation-based performance metrics obtained are-accuracy 94.30 %, sensitivity 97.96 %, specificity 88.01 % and area under the curve (AUC) 0.9298 using XGBoost classifier for detecting psychological stress. Most importantly, the framework also identified two frequency bands in PCG signals with high discriminatory power for psychological stress detection as 270–290 Hz and 380–390 Hz. The elimination of multi-modal data acquisition and analysis makes this approach cost-efficient and reduces computational complexity.

Several Feature Extraction Methods Combined with Near-Infrared Spectroscopy for Identifying the Geographical Origins of Milk.

Milk is a kind of dairy product with high nutritive value. Tracing the origin of milk can uphold the interests of consumers as well as the stability of the dairy market. In this study, a fuzzy direct linear discriminant analysis (FDLDA) is proposed to extract the near-infrared spectral information of milk by combining fuzzy set theory with direct linear discriminant analysis (DLDA). First, spectral data of the milk samples were collected by a portable NIR spectrometer. Then, the data were preprocessed by Savitzky-Golay (SG) and standard normal variables (SNV) to reduce noise, and the dimensionality of the spectral data was decreased by principal component analysis (PCA). Furthermore, linear discriminant analysis (LDA), DLDA, and FDLDA were employed to transform the spectral data into feature space. Finally, the k-nearest neighbor (KNN) classifier, extreme learning machine (ELM) and naïve Bayes classifier were used for classification. The results of the study showed that the classification accuracy of FDLDA was higher than DLDA when the KNN classifier was used. The highest recognition accuracy of FDLDA, DLDA, and LDA could reach 97.33%, 94.67%, and 94.67%. The classification accuracy of FDLDA was also higher than DLDA when using ELM and naïve Bayes classifiers, but the highest recognition accuracy was 88.24% and 92.00%, respectively. Therefore, the KNN classifier outperformed the ELM and naïve Bayes classifiers. This study demonstrated that combining FDLDA, DLDA, and LDA with NIR spectroscopy as an effective method for determining the origin of milk.

Development and validation of peritumoral vascular and intratumoral radiomics to predict pathologic complete responses to neoadjuvant chemotherapy in patients with triple-negative breast cancer

BackgroundTo develop and validate a peritumoral vascular and intratumoral radiomics model to improve pretreatment predictions for pathologic complete responses (pCRs) to neoadjuvant chemoradiotherapy (NAC) in patients with triple-negative breast cancer (TNBC).MethodsA total of 282 TNBC patients (93 in the primary cohort, 113 in the validation cohort, and 76 in The Cancer Imaging Archive [TCIA] cohort) were retrospectively included. The peritumoral vasculature on the maximum intensity projection (MIP) from pretreatment DCE-MRI was segmented by a Hessian matrix-based filter and then edited by a radiologist. Radiomics features were extracted from the tumor and peritumoral vasculature of the MIP images. The LASSO method was used for feature selection, and the k-nearest neighbor (k-NN) classifier was trained and validated to build a predictive model. The diagnostic performance was assessed using the ROC analysis.ResultsOne hundred of the 282 patient (35.5%) with TNBC achieved pCRs after NAC. In predicting pCRs, the combined peritumoral vascular and intratumoral model (fusion model) yields a maximum AUC of 0.82 (95% confidence interval [CI]: 0.75, 0.88) in the primary cohort, a maximum AUC of 0.67 (95% CI: 0.57, 0.76) in the internal validation cohort, and a maximum AUC of 0.65 (95% CI: 0.52, 0.78) in TCIA cohort. The fusion model showed improved performance over the intratumoral model and the peritumoral vascular model, but not significantly (p > 0.05).ConclusionThis study suggested that combined peritumoral vascular and intratumoral radiomics model could provide a non-invasive tool to enable prediction of pCR in TNBC patients treated with NAC.

Student performance prediction employing k-Nearest Neighbor Classification model and meta-heuristic algorithms

Student performance prediction employing k-Nearest Neighbor Classification model and meta-heuristic algorithms

Adaptive three-way KNN classifier using density-based granular balls

The granular ball k-nearest neighbors (GBKNN) algorithm improves the efficiency and robustness of traditional k-nearest neighbors (KNN) by replacing point input with granular ball. However, in the generation process of granular balls by GBKNN, there may be unbalanced distribution of data points existed in some granular balls, which will cause more classification errors. In addition, the fixed value of k in GBKNN may also reduce the accuracy of classification. In order to address these issues, an adaptive three-way KNN classifier using density-based granular balls is proposed. Firstly, an improved density-based granular ball computing using density peak clustering is presented. This method introduces a refined threshold to subdivide the granular balls. Secondly, a data-driven neighborhood is defined to search the optimal k value and a density-based granular ball KNN (DBGBKNN) algorithm is proposed. Thirdly, by considering the fuzziness of the testing set in the classification process, the density-based granular ball KNN with three-way decision (DBGBKNN-3WD) is constructed. Finally, experimental results verify that DBGBKNN-3WD achieves high comprehensive score and low time complexity while maintaining less fuzziness loss than other algorithms.

Evaluation of sequential feature selection in improving the K-nearest neighbor classifier for diabetes prediction

The K-nearest neighbor (KNN) classifier employs distance metrics to measure the distance between the test instance and the samples used in training. With smaller samples, the KNN classifier achieves higher accuracy with low computational time. However, computing the distance between the test instance and all training samples to determine the class of the test instance requires higher computational time for a high-dimensional dataset. This research employs sequential feature selection (SFS) to select the optimal feature for diabetes prediction while reducing the computational time complexity of the KNN classifier. The KNN classifier showed effectiveness with an accuracy rate of 84.41% with nine features. The performance of the KNN improves by 2.6% when trained on the optimal features selected with the SFS. The result revealed glucose level, blood pressure (BP), skin thickness (ST), diabetes pedigree function (DPF), age, and body mass index (BMI) as the most representative features in diabetes prediction. The KNN classifier gives higher accuracy with these features. However, insulin and the number of times a woman is pregnant do not show a significant effect on the KNN classifier.

Comparative analysis of machine learning models for breast cancer prediction and diagnosis: a dual-dataset approach

<p>Breast cancer is ranked as a significant cause of mortality among females globally. Its complex nature poses principal challenges for physicians and researchers for rapid diagnosis and prognosis. Hence, machine learning algorithms are employed to forecast and identify diseases. This study discusses the comparative analysis of seven machine learning models, e.g., logistic regression (LR), support vector machine (SVM), k-nearest neighbor classifier (KNN), decision tree classifier (DT), random forest classifier (RF), Naïve Bayes (NB), and artificial neural network (ANN) to predict breast cancer using Wisconsin breast cancer and breast cancer datasets. In the Wisconsin breast cancer dataset, KNN depicted 99% accuracy, followed by RF (98%), SVM (96%), NB (96%), LR (96%), ANN (93%), and DT (92%). On the contrary, in the breast cancer (BC) dataset, the highest accuracy was achieved by LR at 83%, and the lowest was achieved by DT (65%), which depicted that the numeric dataset WBC has better accuracy than the breast cancer dataset.</p>

Eliminating Negative Word Similarities for Measuring Document Distances: A Thoroughly Empirical Study on Word Mover's Distance.

Document distance is a fundamental yet significant research topic in the information retrieval community, and its accuracy dominates the performance of many text retrieval applications. Beyond the Bag-of-Words (BoW) model, the Word Mover's Distance (WMD) semantically defines the distance between documents as the minimum cost (i.e., measured by word similarities of embeddings) required to transport the words from one document to another, and it has been proven to be superior by k -nearest neighbor classification. In this article, we thoroughly study the characteristics of WMD and its relaxed versions, e.g., Relaxed WMD (RWMD) and Iterative Constrained Transfers (ICT), in various scenarios. Specifically, we concentrate on the problem of negative word similarity: the WMD family leverages all word similarities, however, most of them are meaningless, resulting in negative effects for measuring document distances. To remedy this problem, we propose Informative Similarity Filter (ISF), which retains a very small percentage of top word similarities and fixes the others as the same lower similarity. Built on it, we propose a greedy optimization (GOM) for WMD, an accurate approximation to WMD. We theoretically analyze that ISF-GOM is more applicable for relatively longer documents. Extensive experiments have been conducted to validate: 1) the problem of RWMD; 2) the effectiveness of ISF-GOM; and 3) the consistence of our analysis of ISF-GOM. Our codes and datasets are available at https://github.com/BoCheng-96/ISF-GOM.

Comparative performance analysis of different types of k-nearest neighbor (k-NN) classifiers for fault diagnosis of air compressor setup

This paper presents a method for identifying Reciprocating Air Compressor (RAC) faults using acoustic signals obtained from both healthy and unhealthy conditions. The entire procedure is carried out with Uni-directional microphones using a LABVIEW-based data collection interface and data acquisition (DAQ) hardware unit that has several ports. Accumulated one healthy and seven unhealthy signals of RAC setup processed using signal processing technique called Local Mean Decomposition (LMD). Further, six Statistical Properties (SPs) have been evaluated in order to extract fault features namely: mean (US), variance (σ s2), root square of mean (Mrms), root amplitude of mean (Mrma), absolute amplitude of mean (Mama), and Kurtosis index (Ki). Extracted fault features are classified using various types of k-NN classifiers namely: fine (f-kNN), medium (m-kNN), coarse (c-kNN) and weighted (w-kNN). It has been found that LMD along with 6 statistical properties and form different type of k-NN classifiers, the weighted k-NN classifier has a greater accuracy of 86.74%, which is quite accurate as compared to other k-NN classifiers.

Advanced Ensemble Framework for Diabetes Outcome Forecasting

Abstract: Diabetes is one of the chronic diseases, which is increasing from year to year. Developing an automated system that can detect diabetes patients plays an important role in medical science. A stack classifier model is designed for the detection of diabetes by combining three base estimators such as Random Forest Classifier, LightGBM Classifier, and K-Nearest Neighbors Classifier, and Logistic Regression as meta-classifiers. The data preprocessing includes transforming categorical variables into numerical format. Each base learner is trained on the preprocessed data and predictions are made. In the meta learner stage, Logistic Regression is trained to make predictions based on the predictions of the base learners. The goal of the meta learner is to learn how to combine the predictions of the base learners to make a more accurate final prediction on whether the patient is diabetic or not. The use of a stacking classifier improves prediction accuracy compared to using a single classifier. The developed model gives an accuracy of 98%. The 5-fold cross validation is used to get a more robust estimation of generalization error. Thus, the developed model offers a means to enhance early detection and elevate the quality of care for diabetic patients.

AI-based model for automatic identification of multiple sclerosis based on enhanced sea-horse optimizer and MRI scans

This study aims to develop an AI-enhanced methodology for the expedited and accurate diagnosis of Multiple Sclerosis (MS), a chronic disease affecting the central nervous system leading to progressive impairment. Traditional diagnostic methods are slow and require substantial expertise, underscoring the need for innovative solutions. Our approach involves two phases: initially, extracting features from brain MRI images using first-order histograms, the gray level co-occurrence matrix, and local binary patterns. A unique feature selection technique combining the Sine Cosine Algorithm with the Sea-horse Optimizer is then employed to identify the most significant features. Utilizing the eHealth lab dataset, which includes images from 38 MS patients (mean age 34.1 ± 10.5 years; 17 males, 21 females) and matched healthy controls, our model achieved a remarkable 97.97% detection accuracy using the k-nearest neighbors classifier. Further validation on a larger dataset containing 262 MS cases (199 females, 63 males; mean age 31.26 ± 10.34 years) and 163 healthy individuals (109 females, 54 males; mean age 32.35 ± 10.30 years) demonstrated a 92.94% accuracy for FLAIR images and 91.25% for T2-weighted images with the Random Forest classifier, outperforming existing MS detection methods. These results highlight the potential of the proposed technique as a clinical decision-making tool for the early identification and management of MS.

Flexible wide-range multidimensional force sensors inspired by bones embedded in muscle

AbstractFlexible sensors have been widely studied for use in motion monitoring, human‒machine interactions (HMIs), personalized medicine, and soft intelligent robots. However, their practical application is limited by their low output performance, narrow measuring range, and unidirectional force detection. Here, to achieve flexibility and high performance simultaneously, we developed a flexible wide-range multidimensional force sensor (FWMFS) similar to bones embedded in muscle structures. The adjustable magnetic field endows the FWMFS with multidimensional perception for detecting forces in different directions. The multilayer stacked coils significantly improved the output from the μV to the mV level while ensuring FWMFS miniaturization. The optimized FWMFS exhibited a high voltage sensitivity of 0.227 mV/N (0.5–8.4 N) and 0.047 mV/N (8.4–60 N) in response to normal forces ranging from 0.5 N to 60 N and could detect lateral forces ranging from 0.2–1.1 N and voltage sensitivities of 1.039 mV/N (0.2–0.5 N) and 0.194 mV/N (0.5–1.1 N). In terms of normal force measurements, the FWMFS can monitor finger pressure and sliding trajectories in response to finger taps, as well as measure plantar pressure for assessing human movement. The plantar pressure signals of five human movements collected by the FWMFS were analyzed using the k-nearest neighbors classification algorithm, which achieved a recognition accuracy of 92%. Additionally, an artificial intelligence biometric authentication system is being developed that classifies and recognizes user passwords. Based on the lateral force measurement ability of the FWMFS, the direction of ball movement can be distinguished, and communication systems such as Morse Code can be expanded. This research has significant potential in intelligent sensing and personalized spatial recognition.

AUTOMATED ANAEMIA DETECTION FROM CONJUNCTIVA IMAGES : A MACHINE LEARNING APPROACH FOR ANDROID APPLICATION

Anaemia is a common global disorder condition in which the red blood cell count is lower than normal. Traditional diagnostic methods often prove costly, invasive, and inaccessible, leading to delays in treatment and severe consequences. This study explores non-invasive techniques leveraging smartphone technology for efficient anaemia detection. Some researcher investigated that eye’s conjunctiva analysis can a viable alternative, considering its rich blood vessel network and sensitivity to anaemia indicators, independent of skin color. Utilizing smartphone cameras, the study establishes a robust correlation between the color of the conjunctiva and anaemia status, offering a promising avenue for non-invasive diagnosis. Employing a hybrid methodology, the study integrates You Only Look Once (YOLO) version 8 for efficient object detection, achieving a mean average precision of 96% in extracting Regions of Interest (ROI) from conjunctiva images. Subsequently, K-Nearest Neighbors (KNN) classification of features extracted from these ROI’s attained an 83% accuracy rate in determining anaemia status. Furthermore, the study aims to extend its impact by developing an Android application using the Flutter framework. This application integrates the established YOLO and KNN approaches, enabling real-time anaemia detection through smartphone cameras. Such a tool holds the potential to facilitate early evaluations in resource-constrained regions, enabling timely diagnosis and intervention, thus mitigating the adverse effects of untreated anaemia.

Pattern Recognition of Partial Discharge Faults in Switchgear Using a Back Propagation Neural Network Optimized by an Improved Mantis Search Algorithm.

The dependable functioning of switchgear is essential to maintain the stability of power supply systems. Partial discharge (PD) is a critical phenomenon affecting the insulation of switchgear, potentially leading to equipment failure and accidents. PDs are generally grouped into metal particle discharge, suspended discharge, and creeping discharge. Different types of PDs are closely related to the severity of a PD. Partial discharge pattern recognition (PDPR) plays a vital role in the early detection of insulation defects. In this regard, a Back Propagation Neural Network (BPNN) for PDPR in switchgear is proposed in this paper. To eliminate the sensitivity to initial values of BPNN parameters and to enhance the generalized ability of the proposed BPRN, an improved Mantis Search Algorithm (MSA) is proposed to optimize the BPNN. The improved MSA employs some boundary handling strategies and adaptive parameters to enhance the algorithm's efficiency in optimizing the network parameters of BPNN. Principal Component Analysis (PCA) is introduced to reduce the dimensionality of the feature space to achieve significant time saving in comparable recognition accuracy. The initially extracted 14 feature values are reduced to 7, reducing the BPNN parameter count from 183 with 14 features to 113 with 7 features. Finally, numerical results are presented and compared with Decision Tree (DT), k-Nearest Neighbor classifiers (KNN), and Support Vector Machine (SVM). The proposed method in this paper exhibits the highest recognition accuracy in metal particle discharge and suspended discharge.

Improved Kepler Optimization Algorithm for enhanced feature selection in liver disease classification

Liver diseases represent a significant healthcare challenge, impacting millions globally and posing complexities in diagnosis. To address this global health concern, this paper introduces a groundbreaking enhancement to the Kepler Optimization Algorithm, termed I-KOA, designed specifically for feature selection in high-dimensional datasets. By harnessing the synergies of Opposition-Based Learning and a Local Escaping Operator grounded in the k-nearest Neighbor (kNN) classifier, I-KOA asserts itself as a potent tool for local exploitation, balanced exploration, and evasion of local optima. To our knowledge, this is the first work to exploit KOA as a feature selection method. Pioneering the utilization of KOA as a feature selection method, the paper rigorously tests I-KOA in two extensive experiments, tackling the complex CEC’22 benchmark suite functions and the intricate landscape of five liver disease datasets. Results underscore I-KOA’s unparalleled performance, validated through the Friedman test, where it surpasses seven rival optimization algorithms. Achieving an outstanding overall classification accuracy of 93.46%, Feature selection size of 0.1042, sensitivity of 97.46%, precision of 94.37%, and F1-score of 90.35% across the liver disease datasets, I-KOA’s randomized algorithm ensures robust feature selection, striking a compelling balance between subset size and classification efficacy. Acknowledging computational demands and generalization nuances, I-KOA is a formidable tool ready to revolutionize medical diagnosis and decision support systems. The open source codes of the proposed I-KOA are available at https://www.mathworks.com/matlabcentral/fileexchange/161376-improved-kepler-optimization-algorithm.

CCADD: An online webserver for Alzheimer's disease detection from brain MRI

Alzheimer's disease (AD) imposes a growing burden on public health due to its impact on memory, cognition, behavior, and social skills. Early detection using non-invasive brain magnetic resonance images (MRI) is vital for disease management. We introduce CCADD (Corpus Callosum-based Alzheimer's Disease Detection), a user-friendly webserver that automatically identifies and segments the corpus callosum (CC) region from brain MRI slices. Extracted shape and size-based features of CC are fed into Support Vector Machines (SVM), Random Forest (RF), eXtreme Gradient Boosting (XGBoost), K-Nearest Neighbor (KNN), and Artificial Neural Network (ANN) classifiers to predict AD or Mild Cognitive Impairment (MCI). Exhaustive benchmarking on ADNI data reveals high prediction accuracies for different AD severity levels. CCADD empowers clinicians and researchers for AD detection. This server is available at: http://www.hpppi.iicb.res.in/add.

The Impact of Device Type Number on IoT Device Classification

Today, connected systems are widely used with the recent developments in technology. The internet-connected devices create data traffic when communicating with each other. These data may contain extremely confidential information. Observers can obtain confidential information from the traffic when the security of this traffic cannot be adequately ensured. This confidential information can be personal information as well as information about the type of device used by the person. Attackers could use machine learning to analyze encrypted data traffic patterns from IoT devices to infer sensitive information, even without decrypting the actual content. For example, if someone uses IoT devices for health monitoring or smoke detection, attackers could leverage machine learning to discern victims' habits or identify health conditions. An increase in the number of IoT devices may decrease the accuracy of classification when using machine learning. This paper presents the importance of the effect of device type number on the classification of IoT devices. Therefore, inference attacks on privacy with machine learning algorithms, attacks on machine learning models, and the padding method that is commonly used against such attacks are presented. Moreover, experiments are carried out by using the dataset of the traffic generated by the Internet of Things (IoT) devices. For this purpose, Random Forest, Decision Tree, and k-Nearest Neighbors (k-NN) classification algorithms are compared, and the accuracy rate changes according to the number of devices are presented. According to the results, the Random Forest and Decision Tree algorithms are found to be more effective than the k-NN algorithm. When considering a scenario with two device types, the Random Forest and Decision Tree algorithms achieved an accuracy rate of 98%, outperforming the k-NN algorithm, which had an accuracy rate of 95%.