K-nearest Neighbor Regression Research Articles

Deep Neural Networks Outperform the CAPRA Score in Predicting Biochemical Recurrence After Prostatectomy.

BackgroundUse of predictive models for the prediction of biochemical recurrence (BCR) is gaining attention for prostate cancer (PCa). Specifically, BCR occurs in approximately 20–40% of patients five years after radical prostatectomy (RP) and the ability to predict BCR may help clinicians to make better treatment decisions. We aim to investigate the accuracy of CAPRA score compared to others models in predicting the 3-year BCR of PCa patients.Material and MethodsA total of 5043 men who underwent RP were analyzed retrospectively. The accuracy of CAPRA score, Cox regression analysis, logistic regression, K-nearest neighbor (KNN), random forest (RF) and a densely connected feed-forward neural network (DNN) classifier were compared in terms of 3-year BCR predictive value. The area under the receiver operating characteristic curve was mainly used to assess the performance of the predictive models in predicting the 3 years BCR of PCa patients. Pre-operative data such as PSA level, Gleason grade, and T stage were included in the multivariate analysis. To measure potential improvements to the model performance due to additional data, each model was trained once more with an additional set of post-operative surgical data from definitive pathology.ResultsUsing the CAPRA score variables, DNN predictive model showed the highest AUC value of 0.7 comparing to the CAPRA score, logistic regression, KNN, RF, and cox regression with 0.63, 0.63, 0.55, 0.64, and 0.64, respectively. After including the post-operative variables to the model, the AUC values based on KNN, RF, and cox regression and DNN were improved to 0.77, 0.74, 0.75, and 0.84, respectively.ConclusionsOur results showed that the DNN has the potential to predict the 3-year BCR and outperformed the CAPRA score and other predictive models.

Presenting a soft sensor for monitoring and controlling well health and pump performance using machine learning, statistical analysis, and Petri net modeling.

Groundwater resources play a key role in supplying urban water demands in numerous societies. In many parts of the world, wells provide a reliable and sufficient source of water for domestic, irrigation, and industrial purposes. In recent decades, artificial intelligence (AI) and machine learning (ML) methods have attracted a considerable attention to develop Smart Control Systems for water management facilities. In this study, an attempt has been made to create a smart framework to monitor, control, and manage groundwater wells and pumps using a combination of ML algorithms and statistical analysis. In this research, 8 different learning methods and regressions namely support vector regression (SVR), extreme learning machine (ELM), classification and regression tree (CART), random forest (RF), artificial neural networks (ANNs), generalized regression neural network (GRNN), linear regression (LR), and K-nearest neighbors (KNN) regression algorithms have been applied to create a forecast model to predict water flow rate in Mashhad City wells. Moreover, several descriptive statistical metrics including mean squared error (MSE), root mean square error (RMSE), mean absolute error (MAE), and cross predicted accuracy (CPA) are calculated for these models to evaluate their performance. According to the results of this investigation, CART, RF, and LR algorithms have indicated the highest levels of precision with the lowest error values while SVM and MLP are the worst algorithms. In addition, sensitivity analysis has demonstrated thatthe LR and RF algorithms have produced the most accurate modelsfor deep and shallowwells respectively.Finally, a Petri net model has been presented to illustrate the conceptual model of the smart framework and alarm management system.

Using Soft Sensors as a Basis of an Innovative Architecture for Operation Planning and Quality Evaluation in Agricultural Sprayers

One of the major problems facing humanity in the coming decades is the production of food on a large scale. The production of large quantities of food must be conducted in a sustainable and responsible manner for nature and humans. In this sense, the appropriate application of agricultural pesticides plays a fundamental role since pesticide application in a qualified manner reduces human and environmental risks as well as the costs of food production. Evaluation of the quality of application using sprayers is an important issue, and several quality descriptors related to the average diameter and distribution of droplets are used. This paper describes the construction of a data-driven soft sensor using the parametric principal component regression (PCR) method based on principal component analysis (PCA), which works in two configurations: with the input being the operating conditions of the agricultural boom sprayers and its outputs being the prediction of the quality descriptors of spraying, and vice versa. The soft sensor provides, in one configuration, estimates of the quality of pesticide application at a certain time and, in the other, estimates of the appropriate sprayer-operating conditions, which can be used for control and optimization of the processes in pesticide application. Full cone nozzles are used to illustrate a practical application as well as to validate the usefulness of the soft sensor designed with the PCR method. The selection of historical data, exploration, and filtering of data, and the structure and validation of the soft sensor are presented. For comparison purposes, the results with the well-known nonparametric k-Nearest Neighbor (NN) regression method are presented. The results of this research reveal the usefulness of soft sensors in the application of agricultural pesticides and as a knowledge base to assist in agricultural decision-making.

Advanced Ensemble Model for Solar Radiation Forecasting Using Sine Cosine Algorithm and Newton’s Laws

As research in alternate energy sources is growing, solar radiation is catching the eyes of the research community immensely. Since solar energy generation depends on uncontrollable natural variables, without proper forecasting, this energy source cannot be trusted. For this forecasting, the use of machine learning algorithms is one of the best choices. This paper proposed an optimized solar radiation forecasting ensemble model consisting of pre-processing and training ensemble phases. The training ensemble phase works on an advanced sine cosine algorithm (ASCA) using Newton’s laws of gravity and motion for objects (agents). ASCA uses sine and cosine functions to update the agent’s position/velocity components by considering its mass. The training ensemble model is then developed using the k-nearest neighbors (KNN) regression. The performance of the proposed ensemble model is measured using a dataset from Kaggle (Solar Radiation Prediction, Task from NASA Hackathon). The proposed ASCA algorithm is evaluated in comparison with the Particle Swarm Optimizer (PSO), Whale Optimization Algorithm (WOA), Genetic Algorithm (GA), Grey Wolf Optimizer (GWO), Squirrel Search Algorithm (SSA), Harris Hawks Optimization (HHO), Hybrid Greedy Sine Cosine Algorithm with Differential Evolution (HGSCADE), Hybrid Modified Sine Cosine Algorithm with Cuckoo Search Algorithm (HMSCACSA), Marine Predators Algorithm (MPA), Chimp Optimization Algorithm (ChOA), and Slime Mould Algorithm (SMA). Obtained results of the proposed ensemble model are compared with those of state-of-the-art models, and significant superiority of the proposed ensemble model is confirmed using statistical analysis such as ANOVA and Wilcoxon’s rank-sum tests.

Industrial Food Quality Analysis Using New k-Nearest-Neighbour methods

The problem of predicting continuous scalar outcomes from functional predictors has received high levels of interest in recent years in many fields, especially in the food industry. The k-nearest neighbor (k-NN) method of Near-Infrared Reflectance (NIR) analysis is practical, relatively easy to implement, and becoming one of the most popular methods for conducting food quality based on NIR data. The k-NN is often named k nearest neighbor classifier when it is used for classifying categorical variables, while it is called k-nearest neighbor regression when it is applied for predicting noncategorical variables. The objective of this paper is to use the functional Near-Infrared Reflectance (NIR) spectroscopy approach to predict some chemical components with some modern statistical models based on the kernel and k-Nearest Neighbour procedures. In this paper, three NIR spectroscopy datasets are used as examples, namely Cookie dough, sugar, and tecator data. Specifically, we propose three models for this kind of data which are Functional Nonparametric Regression, Functional Robust Regression, and Functional Relative Error Regression, with both kernel and k-NN approaches to compare between them. The experimental result shows the higher efficiency of k-NN predictor over the kernel predictor. The predictive power of the k-NN method was compared with that of the kernel method, and several real data sets were used to determine the predictive power of both methods.

Quantifying the effects of urban land forms on land surface temperature and modelling the spatial variation using machine learning

This study explores the impact on land surface temperature due to the spatial clustering of urban landforms with normalized difference vegetation index, normalized difference water index and dry bare-soil index. In order to determine the contribution of different land use/land cover classes in affecting the land surface temperature, the contribution index was used for summer and winter seasons. For analyzing the intensity of land surface temperature at the local scale, landscape index was used. Results depicted that the contribution of the source and sink landscapes weakens the intensity of land surface temperature in the winter season. However, the contribution of the source and sink landscape promoted the intensity of land surface temperature in the summer season. Furthermore, this study evaluated the predictive performance of four machine learning models, including K-Nearest Neighbor (K-NN) regression, Neural Networks (NN), Random Trees (RT) regression and Support Vector Machine (SVM) regression for land surface temperature.

Improved gap filling approach and uncertainty estimation for eddy covariance N2O fluxes

Agricultural nitrous oxide (N2O) emissions comprise a majority of the global source of this powerful greenhouse gas. Mitigation approaches for reducing emissions are difficult to evaluate at appropriate field scales because of the substantial effort and expense associated with relatively new technology allowing eddy covariance measurements of N2O fluxes (FN2O). Here we present a new approach for gap filling eddy covariance FN2O and estimating annual uncertainties for a temperate grazed grassland. We tested the potential of using one flux tower to evaluate emissions mitigation options in one paddock relative to an adjacent, unchanged paddock by partitioning data by source footprint contribution. Because of the complexity of spatiotemporal controls on FN2O, we generated a large set of environmental variables and features as input for machine learning algorithms. Inputs were transformed using partial least squares (PLS) decomposition, isolating features with the greatest influence on FN2O. PLS scores were fed to both a neural network (NN) and a locally-weighted k-nearest neighbours (kNN) regression. While the NN and kNN preformed similarly well, kNN regression accounted for the largest proportion of variance (52-72%) and resulted in the lowest bias for each of the three source footprint areas (full footprint and two separated adjacent paddocks, P53 and P54). Annual uncertainty estimates included random measurement uncertainty, accuracy and precision of the gap filling approach, and uncertainty associated with choice of threshold for atmospheric turbulence filtering and footprint contributions. Total N2O emissions for the full footprint, P53, and P54 were 7.4 ±0.35, 7.7 ±0.80, and 6.4 ±0.63 kg N2O-N ha−1, respectively in Year 1, and 6.9 ±0.33, 7.3 ±0.63, and 6.7 ±0.63 kg N2O-N ha−1, respectively in Year 2. These 95% confidence intervals on the annual FN2O suggest that we could detect differences of 10-15% between paddocks at this site when testing mitigation options.

CMC of diverse Gemini surfactants modelling using a hybrid approach combining SVR-DA

Quantitative structure-property relationship (QSPR) technique provides a suitable tool to predict the critical micelle concentration (CMC) of Gemini surfactants from their structure descriptors. In this study, a comparative work was conducted to model the CMC property of 211 diverse Gemini surfactants based on their structural characteristics using linear and non-linear quantitative structure?property relationship models. Least squares model (OLS) and partial least squares (PLS) against k-nearest neighbours regression model (KNN), artificial neural network (ANN) and support vector regression (SVR) have been developed to model the CMC. Molecular descriptors were calculated and screened to remove unsuitable descriptors and improve the learning. Results indicate that the improved performance of support vector regression when the hyper-parameters are optimized using Dragonfly algorithm (SVR-DA) was highly capable of predicting the pCMC (-log CMC) values with an average absolute relative deviation (AARD) of 0.666 and coefficient of determination (R?) of 0.9971 for the global dataset.

Machine learning models for ecological footprint prediction based on energy parameters

The ecological footprint is an excellent tool to better understand the consequences of the human behavior on the environment. The growing need for natural resources emphasizes the necessity of their accurate observation, calculation, and prediction. This paper develops and compares four hybrid machine learning models for predicting the total ecological footprint of consumption based on a set of hyper-parameters predefined by the Bayesian optimization algorithm. In particular, K-nearest neighbor regression (KNNReg), random forest regression (RFR) with 93 trees, and two artificial neural networks (ANNs) with two hidden layers were developed and later compared in terms of their performance. As energy inputs, the primary energy consumption from (1) natural gas sources, (2) coal sources, (3) oil sources, (4) wind sources, (5) solar photovoltaic sources, (6) hydropower sources, (7) nuclear sources, and (8) other renewable sources was used. Additionally, population number has also been used as an input. The models were developed using a set of data that include 1804 instances. The ANNs were modeled using two different activation functions in the hidden layers: ReLU and SPOCU. The performance was evaluated using the mean absolute percentage error (MAPE), mean absolute scaled error (MASE), normalized root-mean-squared error (NRMSE), and symmetric mean absolute percentage error (SMAPE). The results show that KNNReg performs the best with MASE of 0.029, followed by the RFR (0.032), ReLU ANN (0.064), and SPOCU ANN (0.089). Moreover, SMOGN was utilized to produce a synthetic test set which was used to additionally test the best performed model. The performance on the SMOGN set demonstrates good performance (MASE=0.022). Lastly, the best performed model was implemented into a GUI that calculates the ecological footprint based on user inputs.

K-Nearest Neighbors regression for the discrimination of gamma rays and neutrons in organic scintillators

Certain organic scintillators, such as EJ-299 and stilbene, have the ability to distinguish gamma rays and neutrons through the process of pulse shape discrimination (PSD). This work introduces the novel application of a K-Nearest Neighbor (KNN) regressor to improve PSD performance, and applies it to six combinations of photosensors and organic scintillators, with a focus on silicon photomultiplier-based detectors. The method allows for direct comparison against conventional PSD methods by way of a typical PSD Figure of Merit (FOM), while avoiding the uncertainties associated with classification-based machine learning algorithms to achieve an assessment free from predetermined label bias. Tests were conducted to validate the proposed PSD regression method, and parameters such as input features and the number of nearest neighbors used were investigated to maximize the method’s performance. FOM values over a light output range of 200–700 keVee are shown, with improvements ranging from approximately 60% to over 200% when compared to conventional PSD methods. Finally, studies were performed to determine the lower bound of light output at which gamma ray and neutron groupings are still statistically separable. Results indicate that this light output bound can be lowered across all six tested detector-assembly combinations with the proposed KNN regression method when compared to the conventional PSD methods.

Research on Adaptive Selection Algorithm for Multi-model Load Forecasting Based on Adaboost

With the development of energy Internet technology and artificial intelligence algorithms, smart grids have an increasing demand for the accuracy of load forecasting. This paper uses Support Vector Machine (SVM), Artificial Neural Network (ANN) and K-Nearest Neighbors (KNN) regression models to forecast electricity consumption of local residents and wind power generation of selected enterprises respectively. The electricity load of residents is divided into workday load and weekend load for comparative analysis. Based on Adaptive Boosting (Adaboost) algorithm, this paper proposes an adaptive selection method which further improves the prediction accuracy of the regression model, by selecting appropriate number of sub-classifiers and setting training weights of training data and sub-classifiers to adjust the distribution of training samples. Simulation results demonstrate that even in scenarios of poor data quality, the prediction accuracy of the model with the proposed Adaboost-based algorithm can be enhanced by up to 10%.

Soil Classification and Harvest Proposal Implemented using Machine Learning Techniques

The major source of living for the people of India is agriculture. It is considered as important economy for the country. India is one of the country that suffer from natural calamities like drought and flood that may destroy the crops which may lead to heavy loss for the people doing agriculture. Predicting the crop type can help them to cultivate the suitable crop that can be cultivated in that particular soil type. Soil is one major factor or agriculture. There are several types of soil available in our county. In order to classify the soil type we need to understand the characteristics of the soil. Data mining and machine learning is one of the emerging technology in the field of agriculture and horticulture. In order to classify the soil type and Provide suggestion of fertilizers that can improve the growth of the crop cultivated in that particular soil type plays major role in agriculture. For that here exploring Several machine learning algorithms such as Support vector machine(SVM),k-Nearest Neighbour(k-NN) and logistic regression are used to classify the soil type.

A Prediction Model of Blast Furnace Slag Viscosity Based on Principal Component Analysis and K-Nearest Neighbor Regression

Viscosity is considered to be a significant indicator of the metallurgical property of blast furnace (BF) slag. However, a BF is a complicated black box so that the measurement of the viscosity has a large hysteresis. A prediction model for the viscosity based on machine learning, principal component analysis (PCA) and k-nearest neighbor (KNN) regression is presented in this article. First, the main influencing factors of the viscosity are analyzed and selected as the input of the model. Then, the two datasets are preprocessed by data normalization. In addition, the sample characteristics of the data are processed to be statistically irrelevant by PCA. Based on the above, the two datasets are applied to the PCA–KNN model and the support vector regression model, respectively. The results show that the predicted result using the PCA–KNN model is more accurate and reaching 99%.

Learning stacking regression for no-reference super-resolution image quality assessment

No-reference super-resolution (SR) image quality assessment (NR-SRIQA) aims to evaluate the quality of SR images without relying on any reference images. Currently, most previous methods usually utilize a certain handcrafted perceptual statistical features to quantify the degradation of SR images and a simple regression model to learn the mapping relationship from the features to the perceptual quality. Although these methods achieved promising performance, they still have some limitations: 1) the handcrafted features cannot accurately quantify the degradation of SR images; 2) the complex mapping relationship between the features and the quality scores cannot be well approximated by a simple regression model. To alleviate the above problems, we propose a novel stacking regression framework for NR-SRIQA. In the proposed method, we use a pre-trained VGGNet to extract the deep features for measuring the degradation of SR images, and then develop a stacking regression framework to establish the relationship between the learned deep features and the quality scores to achieve the NR-SRIQA. The stacking regression integrates two base regressors, namely Support Vector Regression (SVR) and K-Nearest Neighbor (K-NN) regression, and a simple linear regression as a meta-regressor. Thanks to the feature representation capability of deep neural networks (DNNs) and the complementary features of the two base regressors, the experimental results indicate that the proposed stacking regression framework is capable of yielding higher consistency with human visual judgments on the quality of SR images than other state-of-the-art SRIQA methods.

Cloud spot instance price prediction using kNN regression

Cloud computing can provide users with basic hardware resources, and there are three instance types: reserved instances, on-demand instances and spot instances. The price of spot instance is lower than others on average, but it fluctuates according to market demand and supply. When a user requests a spot instance, he/she needs to give a bid. Only if the bid is not lower than the spot price, user can obtain the right to use this instance. Thus, it is very important and challenging to predict the price of spot instance. To this end, we take the most popular and representative Amazon EC2 as a testbed, and use the price history of its spot instance to predict future price by building a k-Nearest Neighbors (kNN) regression model, which is based on our mathematical description of spot instance price prediction problem. We compare our model with Linear Regression (LR), Support Vector Machine Regression (SVR), Random Forest (RF), Multi-layer Perception Regression (MLPR), gcForest, and the experiments show that our model outperforms the others.

Deep Learning with Loss Ensembles for Solar Power Prediction in Smart Cities

The demand for renewable energy generation, especially photovoltaic (PV) power generation, has been growing over the past few years. However, the amount of generated energy by PV systems is highly dependent on weather conditions. Therefore, accurate forecasting of generated PV power is of importance for large-scale deployment of PV systems. Recently, machine learning (ML) methods have been widely used for PV power generation forecasting. A variety of these techniques, including artificial neural networks (ANNs), ridge regression, K-nearest neighbour (kNN) regression, decision trees, support vector regressions (SVRs) have been applied for this purpose and achieved good performance. In this paper, we briefly review the most recent ML techniques for PV energy generation forecasting and propose a new regression technique to automatically predict a PV system’s output based on historical input parameters. More specifically, the proposed loss function is a combination of three well-known loss functions: Correntropy, Absolute and Square Loss which encourages robustness and generalization jointly. We then integrate the proposed objective function into a Deep Learning model to predict a PV system’s output. By doing so, both the coefficients of loss functions and weight parameters of the ANN are learned jointly via back propagation. We investigate the effectiveness of the proposed method through comprehensive experiments on real data recorded by a real PV system. The experimental results confirm that our method outperforms the state-of-the-art ML methods for PV energy generation forecasting.

Determination of “Hass” Avocado Ripeness During Storage Based on Smartphone Image and Machine Learning Model

Determination of ripeness represented by firmness measured during storage is important to guide the supply chain management of avocados. A machine vision system devised with a smartphone camera was used to capture images. Color values in L*a*b* were extracted from the images. Support vector regression (SVR), K-nearest neighbors regression (KNN), ridge regression, and lasso regression were compared for firmness prediction using the L*a*b* color space values. The results indicated the SVR performed the best among the four machine learning algorithms used. The SVR model predicted firmness of “Hass” avocado with R2, RMSE, and RPD of 0.92, 7.54, and 3.8 respectively for the model validation data set. It was concluded that the machine vision system devised with a smartphone camera and a SVR model could be a low-cost tool for the determination of ripeness of “Hass” avocado during harvest, storage, and distribution.

Auto-3P: An autonomous VNF performance prediction & placement framework based on machine learning

We propose Auto-3P, an Autonomous module for Virtual Network Functions Performance Prediction and Placement at network cloud and edge facilities based on Machine Learning (ML). Auto-3P augments the autonomous placement capabilities of MANagement and Orchestration frameworks (MANOs) by considering both resource availability at hosting nodes and the implied impact of a VNF node placement decisions on the whole service level end-to-end performance. Unlike that, most existing placement methods take a rather myopic approach after manual rule-based decisions, and/or based exclusively on a host-centric view that focuses merely on node-local resource availability and network metrics. We evaluate and validate Auto-3P with real-field trials in the context of a well-defined Smart City Safety use case using a real end-to-end application over a real city-based testbed. We meticulously conduct repeated tests to assess (i) the accuracy of our adopted prediction models; and (ii) their placement performance against three other existing MANO approaches, namely, a “Traditional”, a “Latency-aware” and a “Random” one, as well as against a collection of well-known Time Series Forecasting (TSF) methods. Our results show that the accuracy of our ML models outperforms the one by TSF models, with the most prominent accuracy performances being exhibited by models such as K-Nearest Neighbors Regression (K-NNR), Decision Tree (DT), and Support Vector Regression (SVR). What is more, the resulted end-to-end service level performance of our approach outperforms “Traditional”, “Latency-aware”, and Random MANO placement. Last, Auto-3P achieves load balancing at selected VNF hosts without degrading end-to-end service level delay, and without a need for a (fixed) overload threshold check, unlike what is suggested by other works in the literature for coping with heavy system-wide load conditions.

K-NEAREST NEIGHBOR AND LINEAR REGRESSION IN THE PREDICTION OF THE ARTIFICIAL FORM FACTOR

k-nearest neighbor and linear regression in the prediction of the artificial form factor . T he proposal of this study was to test whether the performance of the non parametric approach k -Nearest Neighbor ( k -NN), would improve estimates of individual artificial form factor ( f 1.3 ) of trees of the hybrid Eucalyptus urophylla x Eucalyptus grandis compared to the O rdinary L east S quares method. A total of 149 sample-trees were selected , felled , and diameter was measured along the trunk at 10% (d 0. 1 ), 30% (d 0. 3 ), 50% (d 0. 5 ) and 70% (d 0. 7 ) of commercial height and posteriorly at 2m intervals . Mathematical models recognized in the literature for predicting the form factor were adjusted for comparison . The hyperparameter k of optimum adjust ment for the k -NN estimator was obtained by repeated cross-validation. The training data of the k -NN regression model were identical to those used in the adjustment of the linear regression models since m ost multiple linear regression models present problems of collinearity or multicollinearity. The use of the covaria te (d0.3.d0.7)/d1.32 and k = 15 made it possible to construct k -NN models with better generalization capacity. The potential of the k -NN estimator to predict the artificial form factor and thus to obtain less biased estimates of individual tree volumes was demonstrated and considered to be superior to the use of linear regression and average form factors. The k -NN approach can be considered more generic for predicti on of the tree form factor , and its use is recommended when classical linear regression models or other simpler methods do not yield good results.

Travel Time Estimation for Destination In Bali Using kNN-Regression Method with Tensorflow

On a tour activity, travel time estimation is needed so that the travel itinerary goes according to the plan. Travel time estimation is very important so we can estimate the time needed to arrive at the destinations in the travel itinerary. Therefore we need a method that can estimate travel time from one place to another. In this study, we propose the k-Nearest Neighbors Regression (kNN-Regression) method with Tensorflow to construct an estimation model. The proposed number of features in our estimation model is 8 features, i.e. zone information, time information, day information, weather information, temperature information, wind speed information, humidity information, and precipitation information. The data obtained from travel information from Ngurah Rai airport to Kuta Beach using GPS and weather information using weather application in real-time. We divide our data into two groups: a historical group consisting of 177 data and a testing group consisting of 51 data. In the testing stage, kNN-Regression will find the historical data closest to the testing data, so that the estimation value of the travel time of some testing data is not much different from the value of the nearest historical data. As a result, our proposed model gives the Mean Absolute Error (MAE) of 2.196078, Root Mean Square Error (RMSE) of 2.977036294 and accuracy rate 88.1819%.