KNN Model Research Articles

Application of machine learning approaches for land cover monitoring in northern Cameroon

Machine learning (ML) models are a leading analytical technique used to monitor, map and quantify land use and land cover (LULC) and its change over time. Models such as k-nearest neighbour (kNN), support vector machines (SVM), artificial neural networks (ANN), and random forests (RF) have been used effectively to classify LULC types at a range of geographical scales. However, ML models have not been widely applied in African tropical regions due to methodological challenges that arise from relying on the coarse-resolution satellite images available for these areas. In this study, we compared the performance of four ML algorithms (kNN, SVM, ANN and RF) applied to LULC monitoring within the Mayo Rey department, North Province, Cameroon. We used satellite data from the Landsat 7 Enhanced Thematic Mapper Plus (ETM+) combined with 8 Operational Land Imager (OLI) images of northern Cameroon for November 2000 and November 2020. Our results showed that all four classification algorithms produced relatively high accuracy (overall classification accuracy >80%), with the RF model (> 90% classification accuracy) outperforming the kNN, SVM, and ANN models. We found that approximately 7% of all forested areas (dense forest and woody savanna) were converted to other land cover types between 2000 and 2020; this forest loss is particularly associated with an expansion of both croplands and built-up areas. Our study represents a novel application and comparison of statistical and ML approaches to LULC monitoring using coarse-resolution satellite images in an African tropical forest and savanna setting. The resulting land cover maps serve as an important baseline that will be useful to the Cameroon government for policy development, conservation planning, urban planning, and deforestation and agricultural monitoring.

Machine Learning in Predicting Lattice Constant of Cubic Perovskite Oxides

A sample of 3,115 data of perovskite oxides in the form of ABO3 (A and B being the cations) was taken for this study of the application of machine learning in predicting the lattice constants (a determining factor in material design). The ANN, DT, RF, KNN, and SVM models were used to predict the lattice constants of perovskites because machine learning techniques have been phenomenal in uncovering crystal structures in the field of material research in recent years. These models used properties like ionic radii, formation energy, and band gap as input features. The R2 score was used to assess the regression model’s performance. The Random Forest Regression Model outperforms all other regression models regarding dataset and features.

Analysis of enhancing drug bioavailability via nanomedicine production approach using green chemistry route: Systematic assessment of drug candidacy

Evaluation of drug candidacy for processing in nanonization is of great importance due to the importance of nanomedicine for enhanced bioavailability approach. Given that the majority of newly discovered drugs are poorly soluble in the body which makes also poor bioavailability of the drug substances, advanced techniques are required in order to enhance the drug efficacy and consequently its solubility. We employed three distinct machine learning models to make predictions in this work, including Multilayer Perceptron regressor (MLP), K-nearest Neighbors regressor (KNN), and Decision Tree Regressor (DT). The regression task has two input features: temperature (K) and pressure (MPa), and the only output is the solubility value of ANA (Anastrozole) drug in supercritical CO2 as the solvent. We tuned all three models using their important hyper-parameters and found the most appropriate combinations for each. Then, many standard measures are used to evaluate their performance. DT, KNN, and MLP models have R2-scores of 0.875, 0.992, and 0.9984, respectively, and have MSE errors of 1.5242, 1.0006 × 10−1, and 2.5688 × 10−2. Additionally, taking into account the MAE, 1.01, 2.62 × 10−1, and 1.17 × 10−1 values acquired. Finally, the multilayer perceptron (MLP) is selected as the most accurate model. Additionally, this model has a maximum error of 3.78 × 10−1 on the dataset.

Improving prediction of chickpea wilt severity using machine learning coupled with model combination techniques under field conditions

Accurate estimation of disease severity in the field is a key to minimize the yield losses in agriculture. Existing disease severity assessment methods have poor accuracy under field conditions. To overcome this limitation, this study used thermal and visible imaging with machine learning (ML) and model combination (MC) techniques to estimate plant disease severity under field conditions. Field experiments were conducted during 2017–18, 2018–19 and 2021–22 to obtain RGB and thermal images of chickpea cultivars with different levels of wilt resistance grown in wilt sick plots. ML models were constructed using four different datasets created using the wilt severity and image derived indices. ML models were also combined using MC techniques to assess the best predictor of the disease severity. Results indicated that the Cubist was the best ML model, while the KNN model was the poorest predictor of chickpea wilt severity under field conditions. MC techniques improved the prediction accuracy of wilt severity over individual ML models. Combining ML models using the least absolute deviation technique gave the best predictions of wilt severity. The results obtained in the present study showed the MC techniques coupled with ML models improved the prediction accuracies of plant disease severity under field conditions.

Rapid and Accurate Prediction of Soil Texture Using an Image-Based Deep Learning Autoencoder Convolutional Neural Network Random Forest (DLAC-CNN-RF) Algorithm

Soil determines the degree of water infiltration, crop nutrient absorption, and germination, which in turn affects crop yield and quality. For the efficient planting of agricultural products, the accurate identification of soil texture is necessary. This study proposed a flexible smartphone-based machine vision system using a deep learning autoencoder convolutional neural network random forest (DLAC-CNN-RF) model for soil texture identification. Different image features (color, particle, and texture) were extracted and randomly combined to predict sand, clay, and silt content via RF and DLAC-CNN-RF algorithms. The results show that the proposed DLAC-CNN-RF model has good performance. When the full features were extracted, a very high prediction accuracy for sand (R2 = 0.99), clay (R2 = 0.98), and silt (R2 = 0.98) was realized, which was higher than those frequently obtained by the KNN and VGG16-RF models. The possible mechanism was further discussed. Finally, a graphical user interface was designed and used to accurately predict soil types. This investigation showed that the proposed DLAC-CNN-RF model could be a promising solution to costly and time-consuming laboratory methods.

Machine Learning on Fault Diagnosis in Wind Turbines

With the improvement in wind turbine (WT) operation and maintenance (O&M) technologies and the rise of O&M cost, fault diagnostics in WTs based on a supervisory control and data acquisition (SCADA) system has become among the cheapest and easiest methods to detect faults in WTs.Hence, it is necessary to monitor the change in real-time parameters from the WT and maintenance action could be taken in advance before any major failures. Therefore, SCADA-driven fault diagnosis in WT based on machine learning algorithms has been proposed in this study by comparing the performance of three different machine learning algorithms, namely k-nearest neighbors (kNN) with a bagging regressor, extreme gradient boosting (XGBoost) and an artificial neural network (ANN) on condition monitoring of gearbox oil sump temperature. Further, this study also compared the performance of two different feature selection methods, namely the Pearson correlation coefficient (PCC) and principal component analysis (PCA), and three hyperparameter optimization methods on optimizing the performance of the models, namely a grid search, a random search and Bayesian optimization. A total of 3 years of SCADA data on WTs located in France have been used to verify the selected method. The results showed the kNN with a bagging regressor, with PCA and a grid search, provides the best R2 score, and the lowest root mean square error (RMSE). The trained model can detect the potential of WT faults at least 4 weeks in advance. However, the proposed kNN model in this study can be trained with the Support Vector Machine hybrid algorithm to improve its performance and reduce fault alarm.

Deep Learning-based and Machine Learning-based Application in Skin Cancer Image Classification

Skin cancer arises from the skin and is capable of invading other parts of the body. The earlier detection of malignant skin lesions effectively helps cure skin disease and prevents fatal skin cancer. As part of AI, machine learning and deep learning can learn the characteristics of input datasets and perform classifications with high accuracy. In this paper, the CNN, KNN, and SVM models are implemented and tested based on the datasets collected from ISIC. The idea of the implementation is to classify the images of skin lesions into benign and malignant. The information of GANs based model is gathered. The results display the accuracy of using these models to classify different types of lesions. And the discussion of the results focuses on the efficiency to implement the machine learning and deep learning models and the accuracy of using them. The goal of the study is to figure out which method is more useful in skin cancer identification. And some of the possible practices are also discussed as future expectations.

Detecting offensive speech in conversational code-mixed dialogue on social media: A contextual dataset and benchmark experiments

The spread of Hate Speech on online platforms is a severe issue for societies and requires the identification of offensive content by platforms. Research has modeled Hate Speech recognition as a text classification problem that predicts the class of a message based on the text of the message only. However, context plays a huge role in communication. In particular, for short messages, the text of the preceding tweets can completely change the interpretation of a message within a discourse. This work extends previous efforts to classify Hate Speech by considering the current and previous tweets jointly. In particular, we introduce a clearly defined way of extracting context. We present the development of the first dataset for conversational-based Hate Speech classification with an approach for collecting context from long conversations for code-mixed Hindi (ICHCL dataset). Overall, our benchmark experiments show that the inclusion of context can improve classification performance over a baseline. Furthermore, we develop a novel processing pipeline for processing the context. The best-performing pipeline uses a fine-tuned SentBERT paired with an LSTM as a classifier. This pipeline achieves a macro F1 score of 0.892 on the ICHCL test dataset. Another KNN, SentBERT, and ABC weighting-based pipeline yields an F1 Macro of 0.807, which gives the best results among traditional classifiers. So even a KNN model gives better results with an optimized BERT than a vanilla BERT model.

A Tropospheric Zenith Delay Forecasting Model Based on a Long Short-Term Memory Neural Network and Its Impact on Precise Point Positioning

Global navigation satellite system (GNSS) signals are affected by refraction when traveling through the troposphere, which result in tropospheric delay. Generally, the tropospheric delay is estimated as an unknown parameter in GNSS data processing. With the increasing demand for GNSS real-time applications, high-precision tropospheric delay augmentation information is vital to speed up the convergence of PPP. In this research, we estimate the zenith tropospheric delay (ZTD) from 2018 to 2019 by static precise point positioning (PPP) using the fixed position mode; GNSS observations were obtained from the National Geomatics Center of China (NGCC). Firstly, ZTD outliers were detected, and data gaps were interpolated using the K-nearest neighbor algorithm (KNN). Secondly, The ZTD differences between the KNN and periodic model were employed as input datasets to train the long short-term memory (LSTM) neural network. Finally, LSTM forecasted ZTD differences and the ZTD periodic signals were combined to recover the final forecasted ZTD results. In addition, the forecasted ZTD results were applied in static PPP as a prior constraint to reduce PPP convergence time. Numerical results show that the average root-mean-square error (RMSE) of predicting ZTD is about 1 cm. The convergence time of the PPP which was corrected by the LSTM-ZTD predictions is reduced by 13.9, 22.6, and 30.7% in the summer, autumn, and winter, respectively, over GPT2-ZTD corrected PPP and unconstrained conventional PPP for different seasons.

Prediction of temperature separation of a nitrogen-driven vortex tube with linear, kNN, SVM, and RF regression models

Prediction of temperature separation of a nitrogen-driven vortex tube with linear, kNN, SVM, and RF regression models

Over-the-Counter Breast Cancer Classification Using Machine Learning and Patient Registration Records.

This study aims to determine the feasibility of machine learning (ML) and patient registration record to be utilised to develop an over-the-counter (OTC) screening model for breast cancer risk estimation. Data were retrospectively collected from women who came to the Hospital Universiti Sains Malaysia, Malaysia for breast-related problems. Eight ML models were used: k-nearest neighbour (kNN), elastic-net logistic regression, multivariate adaptive regression splines, artificial neural network, partial least square, random forest, support vector machine (SVM), and extreme gradient boosting. Features utilised for the development of the screening models were limited to information in the patient registration form. The final model was evaluated in terms of performance across a mammographic density. Additionally, the feature importance of the final model was assessed using the model agnostic approach. kNN had the highest Youden J index, precision, and PR-AUC, while SVM had the highest F2 score. The kNN model was selected as the final model. The model had a balanced performance in terms of sensitivity, specificity, and PR-AUC across the mammographic density groups. The most important feature was the age at examination. In conclusion, this study showed that ML and patient registration information are feasible to be used as the OTC screening model for breast cancer.

Artificial intelligence applied to omics data in liver diseases: Enhancing clinical predictions

Rapid development of biotechnology has led to the generation of vast amounts of multi-omics data, necessitating the advancement of bioinformatics and artificial intelligence to enable computational modeling to diagnose and predict clinical outcome. Both conventional machine learning and new deep learning algorithms screen existing data unbiasedly to uncover patterns and create models that can be valuable in informing clinical decisions. We summarized published literature on the use of AI models trained on omics datasets, with and without clinical data, to diagnose, risk-stratify, and predict survivability of patients with non-malignant liver diseases. A total of 20 different models were tested in selected studies. Generally, the addition of omics data to regular clinical parameters or individual biomarkers improved the AI model performance. For instance, using NAFLD fibrosis score to distinguish F0-F2 from F3-F4 fibrotic stages, the area under the curve (AUC) was 0.87. When integrating metabolomic data by a GMLVQ model, the AUC drastically improved to 0.99. The use of RF on multi-omics and clinical data in another study to predict progression of NAFLD to NASH resulted in an AUC of 0.84, compared to 0.82 when using clinical data only. A comparison of RF, SVM and kNN models on genomics data to classify immune tolerant phase in chronic hepatitis B resulted in AUC of 0.8793-0.8838 compared to 0.6759-0.7276 when using various serum biomarkers. Overall, the integration of omics was shown to improve prediction performance compared to models built only on clinical parameters, indicating a potential use for personalized medicine in clinical setting.

An Effective Data Fusion Model for Detecting the Risk of Transmission Line in Smart Grid

With the rapid prosperity of wireless communication, a new era of smart cities has arisen. During the implementation of smart cities, electricity supply plays a profound role and needs to be guaranteed. To do this, a health model with terminal sensors data is usually adopted to enable the security of the transmission line. However, due to the extreme weather condition and the interference of sensors, some transmitted values are missing, which renders the sensors unreliable. Unfortunately, the existing prediction methods have a lower accuracy and there still exists a gap between the real value and the predicted value. In this article, we propose a novel data fusion model, named as KNN-XGBoost, to predict the missing values. Specifically, in this model, it mainly consists of two parts: 1) KNN stage: a KNN model is used to find <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">${K}$ </tex-math></inline-formula> nearest labels as new attributes and 2) XGBoost: an XGBoost model is applied to predict missing attribute values. The experimental results have demonstrated that our proposed model can improve the accuracy from 1.11% to 4.05%, 0.55% to 1.84%, and 1.10% to 4.04% than the other methods under the three grid health models, respectively.

A Detection Method for Crop Fungal Spores Based on Microfluidic Separation Enrichment and AC Impedance Characteristics.

The timely monitoring of airborne crop fungal spores is important for maintaining food security. In this study, a method based on microfluidic separation and enrichment and AC impedance characteristics was proposed to detect spores of fungal pathogens that cause diseases on crops. Firstly, a microfluidic chip with tertiary structure was designed for the direct separation and enrichment of Ustilaginoidea virens spores, Magnaporthe grisea spores, and Aspergillus niger spores from the air. Then, the impedance characteristics of fungal spores were measured by impedance analyzer in the enrichment area of a microfluidic chip. The impedance characteristics of fungal spores were analyzed, and four impedance characteristics were extracted: absolute value of impedance (abs), real part of impedance (real), imaginary part of impedance (imag), and impedance phase (phase). Finally, based on the impedance characteristics of extracted fungal spores, K-proximity (KNN), random forest (RF), and support vector machine (SVM) classification models were established to classify the three fungal spores. The results showed that the microfluidic chip designed in this study could well collect the spores of three fungal diseases, and the collection rate was up to 97. The average accuracy of KNN model, RF model, and SVM model for the detection of three disease spores was 93.33, 96.44 and 97.78, respectively. The F1-Score of KNN model, RF model, and SVM model was 90, 94.65, and 96.18, respectively. The accuracy, precision, recall, and F1-Score of the SVM model were all the highest, at 97.78, 96.67, 96.69, and 96.18, respectively. Therefore, the detection method of crop fungal spores based on microfluidic separation, enrichment, and impedance characteristics proposed in this study can be used for the detection of airborne crop fungal spores, providing a basis for the subsequent detection of crop fungal spores.

Superhuman performance on sepsis MIMIC-III data by distributional reinforcement learning.

We present a novel setup for treating sepsis using distributional reinforcement learning (RL). Sepsis is a life-threatening medical emergency. Its treatment is considered to be a challenging high-stakes decision-making problem, which has to procedurally account for risk. Treating sepsis by machine learning algorithms is difficult due to a couple of reasons: There is limited and error-afflicted initial data in a highly complex biological system combined with the need to make robust, transparent and safe decisions. We demonstrate a suitable method that combines data imputation by a kNN model using a custom distance with state representation by discretization using clustering, and that enables superhuman decision-making using speedy Q-learning in the framework of distributional RL. Compared to clinicians, the recovery rate is increased by more than 3% on the test data set. Our results illustrate how risk-aware RL agents can play a decisive role in critical situations such as the treatment of sepsis patients, a situation acerbated due to the COVID-19 pandemic (Martineau 2020). In addition, we emphasize the tractability of the methodology and the learning behavior while addressing some criticisms of the previous work (Komorowski et al. 2018) on this topic.

Modeling monthly reference evapotranspiration process in Turkey: application of machine learning methods.

In this study, the predictive power of three different machine learning (ML)-based approaches, namely, multi-gene genetic programming (MGGP), M5 model trees (M5Tree), and K-nearest neighbor algorithm (KNN), for long-term monthly reference evapotranspiration (ET0) prediction were investigated. The input data consist of monthly solar radiation (Rs), maximum air temperature (Tmax), and wind speed (Ws) derived from 163 meteorological stations in Turkey. Different input combinations were created and analyzed. The model's performance was evaluated using criteria such as Nash-Sutcliffe efficiency, Kling-Gupta efficiency, relative root mean squared error, mean absolute percentage error, and determination coefficient. Moreover, Taylor, radar, and boxplot diagrams were created. It was determined that the MGGP model outperformed both the M5Tree and the KNN models. The equation obtained from the MGGP model, for the best-performed combination of Rs-Tmax-Ws, was presented. The best weather conditions were obtained as 0.029 to 31.814MJ/m2, - 5.8 to 45.7°C, and 0.140 to 5.086m/s for Rs, Tmax, and Ws, respectively. It was also found that the Rs was the most potent input variable for ET0 estimation while Ws was the weakest.

Detecting distracted students in educational VR environments using machine learning on eye gaze data

Virtual Reality (VR) has been found useful to improve engagement and retention level of students, for some topics, compared to traditional learning tools such as books, and videos. However, a student could still get distracted and disengaged due to a variety of factors including stress, mind-wandering, unwanted noise, and external alerts. Student eye gaze data could be useful for detecting these distracted students. Gaze data-based visualizations have been proposed in the past to help a teacher monitor distracted students. However, it is not practical for a teacher to monitor a large number of student indicators while teaching. To help filter students based on distraction level, we propose an automated system based on machine learning to classify students based on their distraction level. The key aspects are: (1) we created a labeled eye gaze dataset from an educational VR environment, (2) we propose an automatic system to gauge a student’s distraction level from gaze data, and (3) we apply and compare several classifiers for this purpose. Each classifier classifies distraction, per educational activity section, into one of three levels (low, mid or high). Our results show that Random Forest (RF) classifier had the best accuracy (98.88%) compared to the other models we tested. Additionally, a personalized machine learning model using either RF, kNN, or Extreme Gradient Boosting (XGBoost) model was found to improve the classification accuracy significantly.

Deep Predictive Models Based on IoT and Remote Sensing Big Time Series for Precision Agriculture

Managing time series data generated by the intelligent objects around us requires cleaning and processing techniques, as well as a prediction model with high accuracy and low complexity. This study attempts to fuse climate time series data (Temperature, Humidity, Wind speed and Rainfall) and remote sensing data to predict rice yield. We performed statistical analysis of the data, additive decomposition, and measured the correlation between the different variables. We checked the stationarity of the data by using the Augmented Dickey-Fuller ADF test to apply statistical prediction models based on AutoRegression and moving averages for univariate series such as ARIMA (Autoregressive Integrated Moving Average) and for multivariate series such as Vector AutoRegression VAR and Vector Autoregressive Moving Average VARMAX. In addition, we applied a non-parametric model such as KNearest Neighbors KNN and Recurrent Neural Network models such as Long short-term memory LSTM and Gated recurrent unit GRU for rice yield prediction. The best yield estimation is achieved using LSTM with a Root Mean Square Error RSME error of 0.100. Keywords— Agriculture, ARIMA ,KNN, LSTM, Time series.

머신러닝 분류 모형을 이용한 FIBA 여자농구 아시안컵 대회의 승패 예측 및 요인 분석에 관한 연구

The purpose of this study is to predict match results and analyze win/loss factors by combining big data and machine learning classification models using the box scores of the 2015-2021 women"s basketball Asian Cup tournament. The subject of this study was a total of 200 game records among the records obtained through the official records of the 2015, 2017, 2019, and 2021 Women"s Basketball Asian Cup tournaments, and a total of 22 variables were used to predict win/loss results and analyze win/loss factors. In order to predict the win/loss result of the Women"s Basketball Asian Cup competition, five machine learning classification models are used, KNN, Decision Tree, SVM, Logistic Regression, and Random Forest, and predictive performance by model by predicting win/loss results. were comparatively analyzed. In addition, in order to analyze the factors affecting win/loss, the importance of each factor was analyzed using a random forest classification model. First, when analyzing factors affecting win/loss using box score data, it was considered that total score and efficiency factors should be removed before analysis in order to obtain more accurate factor importance. Second, in the analysis of factors affecting victory and defeat after cleaning dirty data, the number of successful shots (FGM) was found to be the most important factor, followed by the shot success rate (FG%), the two-point success rate (2PTS%), and personal fouls (PF)，interception (STL), and so on. Third, in predicting win-loss results, the logistic regression model showed optimal prediction performance than the KNN, decision tree, SVM, and random forest models, and showed 95% prediction accuracy and 0.95 F1 score.

An insight into the microorganism growth prediction by means of machine learning approaches

Microbial enhanced oil recovery (MEOR) is a well-known oil recovery method that is greatly influenced by the growth and metabolism of the microorganisms. Given the complexities and uncertainties associated with identifying the growth mechanism of microorganism, developing an approach to estimate bacterial concentration versus different factors viz. Salinity, temperature and time is still deemed a challenge. Hence, in this study, seven different machine learning methods namely Artificial Neural Network, Support Vector Machine, Decision Tree, K-nearest Neighbors, Ensemble Learning, Random Forest and Adaptive Boosting are utilized to predict bacterial cell concentration. A databank including 110 data points of bacterial cell concentration entailing the incubation time, salinity, temperature and yeast extract has been collected and used for preparation of these models. Graphical and statistical comparisons are used to analyze the performance and accuracy of each integrated model. The retrieved results revealed that the trained ensemble learning model is the most accurate method in estimating the bacterial growth with correlation coefficient and mean squared error of 0.9163 and 0.0542 on the tested dataset, respectively. Moreover, the KNN model with correlation coefficient and mean squared error of 0.6111 and 0.1192, respectively, is the worst model among the seven estimators. This model has great accuracy in training phase while it is not accurate in validation and testing phase. Due to this fact, it can be concluded that KNN model suffers from overfitting problem. In addition, the impacts of incubation time, yeast extract, temperature and salinity on bacterial cell concentration are also ascertained using sensitivity analysis. It is discerned that the temperature and yeast extract are the most and least effective factors on growth of microorganism, respectively.