Nearest-neighbor Regression Research Articles

Integration of machine learning and CFD for modeling mass transfer in water treatment using membrane separation process

This study aims to assess the efficacy of machine learning models in predicting solute concentration (C) distribution in a membrane separation process, using the input parameters which are spatial coordinates. Computational fluid dynamics (CFD) was employed with machine learning for simulation of process. The models evaluated include Kernel Ridge Regression (KRR), Radius nearest neighbor regression (RNN), K-nearest neighbors (KNN), LASSO, and Multi-Layer Perceptron (MLP). Additionally, Harris Hawks Optimization (HHO) was utilized to fine-tune the hyperparameters of these models. Leading the way is the MLP model, which achieves a remarkable test R2 value of 0.98637 together with very low RMSE and MAE values. Strongness and generalization capacity are shown by its consistent performance on test and training datasets. To conclude, the effectiveness of using machine learning regression methods more especially, KRR, KNN, RNN, LASSO, and MLP in estimating concentration from spatial coordinates was demonstrated in this work. For separation science via membranes where predictive modeling of spatial data is essential, the results offer important new perspectives by developing hybrid model.

Machine learning-based prediction of hazards fine PM2.5 concentrations: a case study of Delhi, India

The air quality of many geographical locations has been deteriorating in the last decades. This deterioration in air quality has affected a very large number of people, and they have been diagnosed with asthma and other respiratory problems. Among various pollutants, PM2.5 is the major cause of numerous health-related problems. Predicting PM2.5 concentration levels using ML models of these dissolved particles might help residents and the government prepare a better prevention and safety plan that can eventually lower the risk factor. The present study is based on predicting the PM2.5 concentration of Delhi by applying various ML models based on meteorological features like wind speed, temperature, humidity, visibility, etc. For prediction of PM2.5, linear regression, decision tree regression, RF regression, KNN regression, and Lasso regression methods were employed in the present study. The model performance was assessed using various parameters, including MAE, MSE, RMSE, and R2 Score. In a comparative study of all regression models, linear regression demonstrated the most favorable outcomes. The model exhibited a superior fit to the data, evidenced by its lowest RMSE value (52.19), outperforming the random forest regression (RMSE = 94.75), K Nearest Neighbor regression (RMSE = 83.93), each of which yielded higher RMSE scores compared to linear regression. Lasso regression (RMSE = 65.20) and decision tree (RMSE = 68.22) also exhibited improved performance following linear regression. The findings of this study advocate for implementing strategies to enforce stringent emission regulations for both industrial operations and vehicular activities. Such measures are imperative for mitigating air pollution levels and subsequently curtailing its adverse impacts on public health within the region. Additionally, this study underscores the necessity for further research endeavours to explore future avenues, with the aim of garnering global attention towards addressing this pressing issue.

Estimation of Rubber Plantation Biomass Based on Variable Optimization from Sentinel-2 Remote Sensing Imagery

The rapid, accurate, and non-destructive estimation of rubber plantation aboveground biomass (AGB) is essential for producers to forecast rubber yield and carbon storage. To enhance the estimation accuracy, an increasing number of remote sensing variables are incorporated into the development of multi-parameter models, which makes its practical application and the potential impact on predictive precision challenging due to the inclusion of non-essential or redundant variables. Therefore, this study systematically evaluated the performance of different parameter combinations derived from Sentinel-2 imagery, using variable optimization approaches with four machine learning algorithms (Random Forest Regression, RF; XGBoost Regression, XGBR; K Nearest Neighbor Regression, KNNR; and Support Vector Regression, SVR) for the estimation of the AGB of rubber plantations. The results indicate that RF achieved the best estimation accuracy (R2 = 0.86, RMSE = 15.77 Mg/ha) for predicting rubber plantation AGB when combined with Boruta-selected variables, outperforming other combinations (variable combinations obtained based on importance ranking, univariate combinations, and multivariate combinations). Our research findings suggest that the consideration of parameter-optimized remote sensing variables is advantageous for improving the estimation accuracy of forest biophysical parameters, when utilizing a large number of parameters for estimation.

New paradigm for watershed model development by coupling machine learning algorithm and mechanistic model

The development of watershed models faces double pressure of data requirements and physical interpretation. The simulation of nonpoint source (NPS) pollution is an important application of watershed model, and the best management practices (BMPs) have attracted wide attention as the main control approach of NPS pollution. In this study, a new paradigm was proposed based on the integration of data-driven and mechanistic methods, taking BMP evaluation as an example. Specifically, comprehensive databases were constructed for filter and retention BMPs by collecting, classifying and analyzing published data. Twelve machine learning algorithms were employed for regression analysis between BMPs efficiency and their influencing factors, while the constructed equations were coupled with physical-based models. A case study was performed in a typical catchment of Chaohu Lake Watershed, China. The results demonstrated total interception area, soil type, and vegetation biomass had significant impacts on BMPs performances, while the multilayer perceptron regression (MLPR), K nearest neighbor regression (KNRD), and extremely randomized tree (ETR) methods had the best performances of nutrients removal. The study generated over ten thousand datasets using mechanistic processes, resulting in more efficient and interpretable BMPs evaluator. Compare to traditional methods, this new paradigm has shown advantages in the model application at the watershed scale, enhancing the BMPs evaluator with a higher level of interpretability by coupling the approach with mechanistic models.

Retracted: A Personalized Recommendation System for English Teaching Resources Based on Multi-K Nearest Neighbor Regression Algorithm

Retracted: A Personalized Recommendation System for English Teaching Resources Based on Multi-<i>K</i> Nearest Neighbor Regression Algorithm

Surface tension of binary and ternary mixtures mapping with ASP and UNIFAC models based on machine learning

Modeling predictions of surface tension for binary and ternary liquid mixtures is difficult. In this work, we propose a machine learning model to accurately predict the surface tension of binary mixtures of organic solvents-ionic liquids and ternary mixtures of organic solvents-ionic liquids–water and analytically characterize the proposed model. In total, 1593 binary mixture data points and 216 ternary mixture data points were collected to develop the machine learning model. The model was developed by combining machine learning algorithms, UNIFAC (UNIversal quasi-chemical Functional group Activity Coefficient) and ASP (Abraham solvation parameter). UNIFAC parameters are used to describe ionic liquids, and ASP is used to describe organic solvents. The effect of each parameter on the surface tension is characterized by SHAP (SHapley Additive exPlanation). We considered support vector regression, artificial neural network, K nearest neighbor regression, random forest regression, LightGBM (light gradient boosting machine), and CatBoost (categorical boosting) algorithms. The results show that the CatBoost algorithm works best, MAE = 0.3338, RMSE = 0.7565, and R2 = 0.9946. The SHAP results show that the surface tension of the liquid decreases as the volume and surface area of the anion increase. This work not only accurately predicts the surface tension of binary and ternary mixtures, but also provides illuminating insight into the microscopic interactions between physical empirical models and physical and chemical properties.

Machine learning the spectral function of a hole in a quantum antiferromagnet

Understanding charge motion in a background of interacting quantum spins is a fundamental problem in quantum many-body physics. The most extensively studied model for this problem is the so-called $t\text{\ensuremath{-}}{t}^{\ensuremath{'}}\text{\ensuremath{-}}{t}^{\ensuremath{'}\ensuremath{'}}\text{\ensuremath{-}}J$ model, where the determination of the parameter ${t}^{\ensuremath{'}}$ in the context of cuprate superconductors is challenging. Here we present a theoretical study of the spectral functions of a mobile hole in the $t\text{\ensuremath{-}}{t}^{\ensuremath{'}}\text{\ensuremath{-}}{t}^{\ensuremath{'}\ensuremath{'}}\text{\ensuremath{-}}J$ model using two machine-learning techniques: $K$-nearest neighbor regression (KNN) and a feed-forward neural network (FFNN). We employ the self-consistent Born approximation to generate a dataset of about $1.3\ifmmode\times\else\texttimes\fi{}{10}^{5}$ spectral functions. We show that, for the forward problem, both methods allow for the accurate and efficient prediction of spectral functions, allowing, e.g., rapid searches through parameter space. Furthermore, we find that for the inverse problem (inferring Hamiltonian parameters from spectra), the FFNN can, but the KNN cannot, accurately predict the model parameters using merely the density of states. Our results suggest that it may be possible to use deep-learning methods to predict materials parameters from experimentally measured spectral functions.

Machine-learning design of graphene nanoribbon waveguide side-coupled absorber

Machine learning is emerging as a new approach that provides more options for solving complex problems involving electromagnetic phenomena. This paper evaluates the application of machine learning to the design of graphene-based absorbers, which is a research challenge. Five machine learning algorithms — [Formula: see text]-nearest neighbor regression (kNN), artificial neural network (ANN), decision tree (DT), extremely randomized trees (ETs) and random forest (RF) — are applied to realize the transmission spectrum prediction and reverse design of a graphene nanoribbon waveguide side-coupled absorber. The results show that all five algorithms are effective, with RF being the most accurate in the inverse design. Compared with previous work, the application of machine learning in the intelligent design of graphene absorbers is evaluated more comprehensively, providing a reference for the selection of machine learning algorithms for future inverse design problems.

Machine-learning-based head impact subtyping based on the spectral densities of the measurable head kinematics

BackgroundTraumatic brain injury can be caused by head impacts, but many brain injury risk estimation models are not equally accurate across the variety of impacts that patients may undergo, and the characteristics of different types of impacts are not well studied. We investigated the spectral characteristics of different head impact types with kinematics classification. MethodsData were analyzed from 3262 head impacts from lab reconstruction, American football, mixed martial arts, and publicly available car crash data. A random forest classifier with spectral densities of linear acceleration and angular velocity was built to classify head impact types (e.g., football, car crash, mixed martial arts). To test the classifier robustness, another 271 lab-reconstructed impacts were obtained from 5 other instrumented mouthguards. Finally, with the classifier, type-specific, nearest-neighbor regression models were built for brain strain. ResultsThe classifier reached a median accuracy of 96% over 1000 random partitions of training and test sets. The most important features in the classification included both low- and high-frequency features, both linear acceleration features and angular velocity features. Different head impact types had different distributions of spectral densities in low- and high-frequency ranges (e.g., the spectral densities of mixed martial arts impacts were higher in the high-frequency range than in the low-frequency range). The type-specific regression showed a generally higher R2 value than baseline models without classification. ConclusionThe machine-learning-based classifier enables a better understanding of the impact kinematics spectral density in different sports, and it can be applied to evaluate the quality of impact-simulation systems and on-field data augmentation.

Evaluating parallelized support vector regression and nearest neighbor regression with different input variations for estimating daily global solar radiation of the humid subtropical region in China

AbstractIndirectly estimating global solar radiation is strongly nonlinear and needs to be addressed by machine learning. Sequentially developing a machine learning model can be very time consuming. Moreover, whether and how the exogenous meteorological, geographical and temporal variables affect regression accuracy still has not been well understood. This paper evaluates parallelized support vector regression (SVR) and nearest neighbor regression (NNR) models for estimating daily global solar radiation of the humid subtropical region in China using existing Python libraries on a multi-core central processing unit (CPU) and a graphical processing unit (GPU). Seven input variations are studied. Two variations are commonly adopted in literature, four variations contain meteorological, geographical and/or temporal features with bounded Pearson correlation coefficients (PCCs), and the other variation simply include all the available features. Experimental results demonstrate that: SVR and NNR are equally powerful for nonlinear regression, and the variation comprising features with absolute PCCs no less than 0.3 (i.e. just all the meteorological features) is able to achieve most accurate estimation; the GPU-parallelized SVR model can accelerate parameter calibration and prediction; compared with the CPU-parallelized and GPU-parallelized SVR models, the GPU-parallelized NNR model is much more efficient and rather more scalable with the increment of the number of data samples; and the CPU-parallelized NNR model consumes quite less parameter calibration time than the GPU-parallelized NNR model, owing to different methods adopted for determining distances and significant time wasted by the GPU-parallelized NNR model on repeatedly calculating required information during cross-validation.

Estimation Method of State of Charge of Power Battery of New Energy Vehicle Based on K Nearest Neighbor Regression

In order to improve the power battery state of charge detection ability of new energy vehicles and reduce harmonic interference, a measurement method of power battery state of new energy vehicles based on K-nearest neighbor regression analysis is proposed. Build a charging state parameter collection model of new energy vehicle power battery, consider the unbalanced power of uncertain factors, realize the oscillation and fusion processing of charging state of new energy vehicle power battery through the dynamic matching method of mobile energy storage resources of electric vehicle, extract the microgrid load fluctuation characteristic value of charging state signal of new energy vehicle power battery, and analyze the percentage of remaining electricity to rated capacity. The initial charge and capacity characteristics of the battery of the new energy vehicle power battery are matched. The K-nearest neighbor regression analysis method is used to realize the regression analysis and dynamic estimation of the battery state of charge parameters. According to the state of charge of the new energy vehicle power battery, the steady-state characteristic quantity is output, and the demand response load model parameter reconstruction and load balance matching method are used to realize the state of charge estimation and parameter adaptive evaluation of the new energy vehicle power battery. The simulation results show that the accuracy of measuring the state of charge of the power battery of new energy vehicles is high, and the harmonic suppression ability is strong, which improves the balanced configuration and load balancing scheduling ability of the power system of the power battery of new energy vehicles.

Finding Similar Historical Scenarios for Better Understanding Aircraft Taxi Time: A Deep Metric Learning Approach

Accurate prediction of aircraft taxi time is crucial to optimizing route scheduling and improving airport efficiency. However, studies usually give only a single deterministic taxi time predicted by models, which lacks interpretability and affects controllers’ trust. To solve this problem, using the idea of auxiliary decision-making based on similar scenarios, a deep metric learning model is proposed to learn the similarity among historical situations, consisting of flight properties, surface traffic, and meteorological conditions. Combining the model with the <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">k</i> -nearest-neighbor regression algorithm, a set of historical circumstances similar to a reference scenario can be found and used to predict the taxi time. Experimental verification using Shanghai Pudong International Airport data shows that the model achieves better performance than the state-of-the-art variants for taxi-out time prediction accuracy. More importantly, similar historical scenarios can provide insights into the uncertainty quantification of aircraft taxi time and serve as a basis for controllers to make decisions. The methodology enriches decision support tools and enhances airports’ refined management capabilities.

Toward learning robust contrastive embeddings for binaural sound source localization.

Recent deep neural network based methods provide accurate binaural source localization performance. These data-driven models map measured binaural cues directly to source locations hence their performance highly depend on the training data distribution. In this paper, we propose a parametric embedding that maps the binaural cues to a low-dimensional space where localization can be done with a nearest-neighbor regression. We implement the embedding using a neural network, optimized to map points that are close to each other in the latent space (the space of source azimuths or elevations) to nearby points in the embedding space, thus the Euclidean distances between the embeddings reflect their source proximities, and the structure of the embeddings forms a manifold, which provides interpretability to the embeddings. We show that the proposed embedding generalizes well in various acoustic conditions (with reverberation) different from those encountered during training, and provides better performance than unsupervised embeddings previously used for binaural localization. In addition, the proposed method performs better than or equally well as a feed-forward neural network based model that directly estimates the source locations from the binaural cues, and it has better results than the feed-forward model when a small amount of training data is used. Moreover, we also compare the proposed embedding using both supervised and weakly supervised learning, and show that in both conditions, the resulting embeddings perform similarly well, but the weakly supervised embedding allows to estimate source azimuth and elevation simultaneously.

The Prediction of Sinter Drums Strength Using Hybrid Machine Learning Algorithms.

The prediction model with the sinter drum strength as the evaluation index was established based on the index data and historical sintering data generated during the sintering process. The regression prediction model in the algorithm of machine learning was applied to the prediction of the strength of the sinter drum. After verifying the feasibility of drum strength prediction, different data preprocessing methods were used to preprocess the data. Ten regression prediction algorithms such as linear regression, ridge regression, regression tree, support vector regression, and nearest neighbor regression were used for predicting the sinter drum strength to obtain preliminary prediction results. By comparing the prediction results, the most suitable combinations of data preprocessing algorithms and prediction algorithms for sinter drum strength prediction is obtained. The prediction results show that, for the drum strength of the sinter, using the function data standardization algorithm for data preprocessing has the best effect. Then, using gradient boosting regression, random forest regression, and extra tree regression prediction algorithms resulted in higher prediction accuracy. On this basis, the regression prediction model algorithm parameters are optimized and improved. The parameters of the regression prediction algorithm that are most suitable for the prediction of sinter drum strength are obtained.

Word2vec neural model-based technique to generate protein vectors for combating COVID-19: a machine learning approach.

The world was ambushed in 2019 by the COVID-19 virus which affected the health, economy, and lifestyle of individuals worldwide. One way of combating such a public health concern is by using appropriate, rapid, and unbiased diagnostic tools for quick detection of infected people. However, a current dearth of bioinformatics tools necessitates modeling studies to help diagnose COVID-19 cases. Molecular-based methods such as the real-time reverse transcription polymerase chain reaction (rRT-PCR) for detecting COVID-19 is time consuming and prone to contamination. Modern bioinformatics tools have made it possible to create large databases of protein sequences of various diseases, apply data mining techniques, and accurately diagnose diseases. However, the current sequence alignment tools that use these databases are not able to detect novel COVID-19 viral sequences due to high sequence dissimilarity. The objective of this study, therefore, was to develop models that can accurately classify COVID-19 viral sequences rapidly using protein vectors generated by neural word embedding technique. Five machine learning models; K nearest neighbor regression (KNN), support vector machine (SVM), random forest (RF), Linear discriminant analysis (LDA), and Logistic regression were developed using datasets from the National Center for Biotechnology. Our results suggest, the RF model performed better than all other models on the training dataset with 99% accuracy score and 99.5% accuracy on the testing dataset. The implication of this study is that, rapid detection of the COVID-19 virus in suspected cases could potentially save lives as less time will be needed to ascertain the status of a patient.

Parameter-free surrounding neighborhood based regression methods

In machine learning, nearest neighbor (NN) regression is one of the most prominent methods for numeric prediction. It estimates the output variable of a new data point by averaging the output variables of the neighboring points. The selection of the neighborhood and its parameter(s) is crucial for the performance of NN regression, however this is still an open issue. This study contributes to the literature by adopting the parameter-free surrounding neighborhood (PSN) concept for NN regression. PSNs are based on proximity graphs, i.e. minimum spanning tree, relative neighborhood graph, and Gabriel graph. They yield a unique neighborhood for each point by combining proximity, connectivity and spatial distribution. The performances of the PSN regression methods are compared with k-nearest neighbors, k-nearest centroid neighbors, and support vector regression using real-world data sets. The statistical tests show that the PSN regression methods perform significantly better than most of the competing approaches. Also, the proposed approaches do not have any parameters to be set.

Big data-oriented wheel position and geometry calculation for cutting tool groove manufacturing based on AI algorithms

Groove is a key structure of high-performance integral cutting tools. It has to be manufactured by 5-axis grinding machine due to its complex spatial geometry and hard materials. The crucial manufacturing parameters (CMP) are grinding wheel positions and geometries. However, it is a challenging problem to solve the CMP for the designed groove. The traditional trial-and-error or analytical methods have defects such as time-consuming, limited-applying, and low accuracy. In this study, the problem is translated into a multiple output regression model of groove manufacture (MORGM) based on the big data technology and AI algorithms. The inputs are 34 groove geometry features, and the outputs are 5 CMP. Firstly, two groove machining big data sets with different range are established, each of which is includes 46,656 records. They are used as data resource for MORGM. Secondly, 7 AI algorithms, including linear regression, k nearest-neighbor regression, decision trees, random forest regression, support vector regression, and ANN algorithms, are discussed to build the model. Then, 28 experiments are carried out to test the big data set and algorithms. Finally, the best MORGM is built by ANN algorithm and the big data set with a larger range. The results show that CMP can be calculated accurately and conveniently by the built MORGM.

Machine Learning Approach to Simulate Soil CO2 Fluxes under Cropping Systems

With the growing number of datasets to describe greenhouse gas (GHG) emissions, there is an opportunity to develop novel predictive models that require neither the expense nor time required to make direct field measurements. This study evaluates the potential for machine learning (ML) approaches to predict soil GHG emissions without the biogeochemical expertise that is required to use many current models for simulating soil GHGs. There are ample data from field measurements now publicly available to test new modeling approaches. The objective of this paper was to develop and evaluate machine learning (ML) models using field data (soil temperature, soil moisture, soil classification, crop type, fertilization type, and air temperature) available in the Greenhouse gas Reduction through Agricultural Carbon Enhancement network (GRACEnet) database to simulate soil CO2 fluxes with different fertilization methods. Four machine learning algorithms—K nearest neighbor regression (KNN), support vector regression (SVR), random forest (RF) regression, and gradient boosted (GB) regression—were used to develop the models. The GB regression model outperformed all the other models on the training dataset with R2 = 0.88, MAE = 2177.89 g C ha−1 day−1, and RMSE 4405.43 g C ha−1 day−1. However, the RF and GB regression models both performed optimally on the unseen test dataset with R2 = 0.82. Machine learning tools were useful for developing predictors based on soil classification, soil temperature and air temperature when a large database like GRACEnet is available, but these were not highly predictive variables in correlation analysis. This study demonstrates the suitability of using tree-based ML algorithms for predictive modeling of CO2 fluxes, but no biogeochemical processes can be described with such models.

Reweighting samples under covariate shift using a Wasserstein distance criterion

Considering two random variables with different laws to which we only have access through finite size iid samples, we address how to reweight the first sample so that its empirical distribution converges towards the true law of the second sample as the size of both samples goes to infinity. We study an optimal reweighting that minimizes the Wasserstein distance between the empirical measures of the two samples, and leads to an expression of the weights in terms of Nearest Neighbors. The consistency and some asymptotic convergence rates in terms of expected Wasserstein distance are derived, and do not need the assumption of absolute continuity of one random variable with respect to the other. These results have some application in Uncertainty Quantification for decoupled estimation and in the bound of the generalization error for the Nearest Neighbor Regression under covariate shift.

Denoising modulo samples: k-NN regression and tightness of SDP relaxation

Abstract Many modern applications involve the acquisition of noisy modulo samples of a function $f$, with the goal being to recover estimates of the original samples of $f$. For a Lipschitz function $f:[0,1]^d \to {{\mathbb{R}}}$, suppose we are given the samples $y_i = (f(x_i) + \eta _i)\bmod 1; \quad i=1,\dots ,n$, where $\eta _i$ denotes noise. Assuming $\eta _i$ are zero-mean i.i.d Gaussian’s, and $x_i$’s form a uniform grid, we derive a two-stage algorithm that recovers estimates of the samples $f(x_i)$ with a uniform error rate $O\big(\big(\frac{\log n}{n}\big)^{\frac{1}{d+2}}\big)$ holding with high probability. The first stage involves embedding the points on the unit complex circle, and obtaining denoised estimates of $f(x_i)\bmod 1$ via a $k$NN (nearest neighbor) estimator. The second stage involves a sequential unwrapping procedure which unwraps the denoised mod $1$ estimates from the first stage. The estimates of the samples $f(x_i)$ can be subsequently utilized to construct an estimate of the function $f$, with the aforementioned uniform error rate. Recently, Cucuringu and Tyagi proposed an alternative way of denoising modulo $1$ data, which works with their representation on the unit complex circle. They formulated a smoothness regularized least squares problem on the product manifold of unit circles, where the smoothness is measured with respect to the Laplacian of a proximity graph $G$ involving the $x_i$’s. This is a nonconvex quadratically constrained quadratic program (QCQP) hence they proposed solving its semidefinite program (SDP) based relaxation. We derive sufficient conditions under which the SDP is a tight relaxation of the QCQP. Hence under these conditions, the global solution of QCQP can be obtained in polynomial time.