Nonparametric Regression Algorithm Research Articles

Application of non parametric Bayesian methods in high dimensional data

With the development of technology and the widespread collection of data, high-dimensional data analysis has become a research hotspot in many fields. Traditional parameter methods often face problems such as dimensional disasters in high-dimensional data analysis. Non parametric methods have broad application prospects in high-dimensional data because they do not rely on specific parameter distribution assumptions. The Bayesian rule is more suitable for dealing with noise and outliers in high-dimensional data because it takes uncertainty into account. Therefore, it is of great significance to combine non parametric methods with Bayesian methods for application research in high-dimensional data analysis. In this paper, the nonparametric Bayesian method was applied to the analysis of high-dimensional data, and the Dirichlet process Mixture model was used to cluster high-dimensional data. The regression analysis of high-dimensional data was carried out through the prediction model of nonparametric Bayesian regression. In this paper, the nonparametric Bayesian method based on Bayesian sparse linear model was used for feature selection of high-dimensional data. In order to determine the superiority of nonparametric Bayesian methods in high-dimensional data analysis, this paper conducted experiments on nonparametric Bayesian methods and traditional parametric methods in high-dimensional data analysis from five aspects of cluster analysis, classification analysis, regression analysis, feature selection and anomaly detection, and evaluated them through multiple indicators. This article explored the application of non parametric Bayesian methods in high-dimensional data analysis from these aspects through simulation experiments. The experimental results show that the clustering accuracy of the non parametric Bayesian clustering algorithm was 0.93, and the accuracy of the non parametric Bayesian classification algorithm was between 0.93 and 0.99; the coefficient of determination of nonparametric Bayesian regression algorithm was 0.98; the F1 values of non parametric Bayesian methods in anomaly detection ranged from 0.86 to 0.91, which was superior to traditional methods. Non parametric Bayesian methods have broad application prospects in high-dimensional data analysis, and can be applied in multiple fields such as clustering, classification, regression, etc.

The role of the Lendtech sector in the consumer credit market in the context of household financial exclusion

Research background: According to the World Bank (2020), about 60% of adults in developing countries do not use formal financial services. Furthermore, according to the Polish Association of Loan Institutions (2022), about 3 million Poles use loans, most of them obtained online. Among the reasons for more than a decade of growth of interest in the non-bank consumer lending market there are the development of modern technology applications in finance and the establishment of the Lendtech sector. Purpose of the article: The main goal of the paper is to verify the role played by the Lendtech (LT) sector in the consumer credit market in the context of household financial exclusion. The following research questions were asked: Do credit-excluded households take advantage of LT services and, if so, to what extent? What are the behaviours and preferences of those who use consumer credit offered by LT? Do socio-demographic characteristics determine consumer use of loans offered by LT and, if so, what are they? Is the use of loans offered by LT due to credit exclusion or other factors? What action should be taken by participants in the digital consumer loan market interested in its inclusive direction? Methods: The paper uses the following methods: critical analysis of the literature, Kruskal-Wallis test, Mann-Whitney test, and nonparametric regression algorithm: k-nearest neighbors, as well as inductive inference methods. The data used is primary in nature and comes from a nationwide survey, September 2022 (CAWI method) of 1,200 Poles, of whom 200 respondents are Lendtech customers. The quota selection applied made it possible to reflect characteristics corresponding to the population of customers of lending institutions registered in BIK databases. Findings & value added: The article is a pioneering study based on an independent scientific survey, devoted to the Polish LT services market considered in terms of its relationship with one of the types of financial exclusion: credit exclusion. The most important conclusion is that people at risk of credit exclusion find a financing substitute in the LT sector, and thus it plays an important role in reducing financial exclusion, while maintaining the principle of creditworthiness verification.

Principal component analysis–multivariate adaptive regression splines (PCA-MARS) and back propagation-artificial neural network (BP-ANN) methods for predicting the efficiency of oxidative desulfurization systems using ATR-FTIR spectroscopy

Oxidative desulfurization (ODS) of diesel fuels has received attention in recent years due to mild working conditions and effective removal of the aromatic sulfur compounds. There is a need for rapid, accurate, and reproducible analytical tools to monitor the performance of ODS systems. During the ODS process, sulfur compounds are oxidized to their corresponding sulfones which are easily removed by extraction in polar solvents. The amount of extracted sulfones is a reliable indicator of ODS performance, showing both oxidation and extraction efficiency. This article studies the ability of a non-parametric regression algorithm, principal component analysis-multivariate adaptive regression splines (PCA-MARS) as an alternative to back propagation artificial neural network (BP-ANN) to predict the concentration of sulfone removed during the ODS process. Using PCA, variables were compressed to identify principal components (PCs) that best described the data matrix, and the scores of such PCs were used as input variables for the MARS and ANN algorithms. Thecoefficientofdeterminationincalibration (R2c), root mean square error of calibration (RMSEC) and root mean square error of prediction (RMSEP) were calculated for PCA-BP-ANN (R2c = 0.9913, RMSEC = 2.4206 and RMSEP = 5.7124) and PCA-MARS (R2c = 0.9841, RMSEC = 2.7934 and RMSEP = 5.8476) models and were compared with the genetic algorithm partial least squares (GA-PLS) (R2c = 0.9472, RMSEC = 5.5226 and RMSEP = 9.6417) and as the results reveal, both methods are better than GA-PLS in terms of prediction accuracy. The proposed PCA-MARS and PCA-BP-ANN models are robust models that provide similar predictions and can be effectively used to predict sulfone containing samples. The MARS algorithm builds a flexible model using simpler linear regression and is computationally more efficient than BPNN due to data-driven stepwise search, addition, and pruning.

Evaluating semi- and nonparametric regression algorithms in quantifying stem taper and volume with alternative test data selection strategies

Abstract Accurately quantifying stem taper is essential to predict diameter at any given height along the stem and to estimate tree volume for various sections of the stem. With increased computing power, semi- and nonparamatric methods have been proposed as alternative approaches for modelling tree taper. The main objective of this study was to assess the accuracy of stem taper predicted for four pine and four hardwood species by semi- and nonparametric models. Specifically, generalized additive models (GAM), random forests (RF) and regression-enhanced random forests (RERF) were compared with two widely-used parametric models: variable-exponent function (VAR) and segmented polynomial regression model (SEG). Test datasets selected from four different data splitting methods were used to examine the prediction accuracy of the models. Results showed that all examined models can be used to quantify stem taper and volume for all species when prediction is limited to be within the range of tree sizes used in model building. The nonparametric RF algorithm generally produced higher bias and lower precision than the semiparametric (GAM and RERF) and parametric models (VAR and SEG). For all species, GAM, VAR and SEG provided more robust predictions of stem taper than RF and RERF algorithms, especially when small or large trees were withheld for model testing. The data splitting strategies used in this work provide an efficient ‘stress test’ to evaluate model performance when collecting an independent test dataset is not feasible. The findings of this work provide additional insights for forest practitioners and resource managers to select appropriate methods in stem taper modelling.

Optimal development for a 3D-printed gripper for biomedical and micromanipulation applications by non-parametric regression-based metaheuristic technique

With the advancement of bioengineering and robotic engineering, medical robots have been increasing concern about manipulating the microobject or cells. Although the rigid robots have a stable operation, they inherit many limitations such as the complex assembly process of joints-coupled rigid links and expensive costs. Especially, clearances between kinematic joints cause vibrations that damage microobject. To cope with such problems, a flexure-based polylactic acid (PLA) gripper is developed to realize precise motion in medical robots. The proposed gripper is 3D printed by fused deposition modeling technique with advantages of monolithic structure, jointless and cheap cost. Prior to the gripper fabrication, the optimization development of the gripper is conducted employing a combination of the non-parametric regression (NPR) and multi-objective genetic algorithm (MOGA). Numerical data samples are collected by the finite element method. The modeling results were well formulated utilizing the NPR method with the R2 value greater than 0.9. The Pareto-optimum design results identified that the gripper can provide a high displacement of 2 mm and a small stress of 41 MPa via MOGA. Additionally, the proposed flexure-based compliant PLA gripper can work with a safety factor higher than 1.6. The experiment tests on the prototype of the gripper are close to the estimated values.

Effective combination of modeling and experimental data with deep metric learning for guided wave-based damage localization in plates

Data-driven methods have emerged as promising approaches for guided wave-based damage localization. The success of the methods highly depends on the amount and the quality of the training data. Modeling training data are easy to obtain and abundant but could be unrealistic with biased model parameters and experimental uncertainties unconsidered. Experimental training data are realistic but scanty due to the high cost and low accessibility. To tackle this problem, this work utilizes both types of data to collect sufficient training data and learn a mapping function invariant to the modeling errors and experimental noises in a supervised manner, and thus to improve localization accuracy. Specifically, deep metric learning is conducted on the hybrid data to identify a distance metric, based on which the discrepancy between two sets of data is minimized, i.e., the distance metric is insensitive to the superfluous variations caused by modeling errors and experimental noises. Then, a representative non-parametric regression algorithm that relies on the distance metric, kernel regression, is used to predict damage locations. The performance of the proposed method is demonstrated on damage localization in plates and compared with several other data-driven methods. Results show that that proposed method can serve as a general strategy to overcome the shortage of labelled training data in data-driven methods for damage localization.11The data and code related to this article can be found at https://github.com/amazingheart/TensorFlow-implementation-of-deep-metric-learning.

A Non-Parametric Constrained Regression Algorithm for Dual Norm Computation and its Applications

A Non-Parametric Constrained Regression Algorithm for Dual Norm Computation and its Applications

Fingerprinting-assisted UWB-based localization technique for complex indoor environments

Among the numerous radio-based solutions for indoor localization, ultra-wideband (UWB) technology is of particular interest due to its signal characteristics. The wide bandwidth of the UWB signal provides a fine time resolution of the transmitted pulses that enables a centimetre-level ranging accuracy under line-of-sight (LOS) conditions even in multipath intensive indoor environments. Nevertheless, it is still a challenge to implement accurate UWB-based localization in complex multi-room indoor environments at low cost because of a large number of static UWB anchors that may need to be deployed in order to provide an adequate LOS coverage in every segment of the environment. Therefore, there is a strong interest in developing UWB-based localization techniques that will provide acceptable accuracy under partially LOS coverage conditions. In this paper, we present a novel hybrid method that combines two conventional localization techniques, trilateration and fingerprinting, to address the problem of cost-effective UWB-based localization in complex indoor environments. With the proposed method, the target location is determined by a trilateration algorithm, while a fingerprinting-based algorithm is used to provide additional distances for trilateration in cases when there is an insufficient number of available LOS measurements. The additional distances are generated by a non-parametric regression algorithm that relies on a fingerprint database to map all available online range measurements (LOS as well non-LOS) to distances between the target and the set of pre-defined reference points. To minimize human effort in fingerprint collection, the indoor environment is site-surveyed in a room-by-room fashion with auxiliary UWB anchors temporarily placed at up to three reference points in the surveyed room. The method is validated through an extensive indoor measurement campaign with commercially available UWB transceivers. The experimental results show that the proposed method achieves sub-decimetre level localization accuracy under typical real-world conditions.

Nonparametric Regression Algorithm for Short-Term Traffic Flow Forecasting Based on Data Reduction and Support Vector Machine

Nonparametric regression is an important method for short-term traffic flow forecasting, but the traditional nonparametric regression method needs a large storage space and slow query speed when the data are large and the dimension is high. In this paper, an improved nonparametric regression traffic flow forecasting algorithm is proposed. Subtraction fuzzy clustering method is used to cluster historical data to reduce the amount of data in the pattern database. Principal component analysis (PCA) is used to reduce the dimension of the pattern to overcome the problems of slow matching speed and interference of irrelevant dimension caused by the high dimension of the pattern. The support vector machine method is used to estimate the value of the final predicted variables by searching the patterns. The operation efficiency and prediction accuracy of the algorithm are improved. An online simulation-based test shows that the algorithm exhibits better efficiency and accuracy compared with traditional methods.

An interactive nonparametric evidential regression algorithm with instance selection

The nonparametric evidential regression (EVREG) method provides flexible forms of prediction regarding the value of output, allowing the output of training instances to be partially unknown. However, the superfluous training instances still have negative effects on the parameter learning in EVREG. To relax this limitation, this paper introduces an interactive nonparametric evidential regression (IEVREG) algorithm with instance selection. More specifically, the significance of an instance is firstly measured by defining the evaluation functions, taking into account both the prediction accuracy of regression model and the spatial information between that instance with other ones. According to a search strategy, the instances with high degree of significance are then selected to maximize an objective function. Different from existing instance selection methods, the selection of training instances is synchronously accomplished with the parameter learning in IEVREG, rather than just a separated data preprocessing operation as traditional methods do. Furthermore, the noise and redundant instances can be simultaneously removed and the performance of IEVREG is robust to the order of presentation of instances in raw data set. Experimental results show that the proposed IEVREG algorithm has appropriate prediction accuracy, while performing well selection of the representative training instances from the raw data set. Simulations on synthetic and UCI real-world data sets validate our conclusions.

RESPONSE SURFACE BASED OPTIMIZATION OF RIBBED ISOSCELES TRIANGULAR TWISTED TAPE HEAT EXCHANGER USING ENTROPY AUGMENTATION GENERATION NUMBER WITH AL2O3 NANO WORKING FLUID

In this work, a combination of passive techniques like providing ribs on the duct surface, inserting twisted tapes were employed at different configurations and analysed using ANSYS Fluent 17.2. The enhancement is probed by placing ribs on the duct surfaces at various angles. Twisted tape inserts were used in conjunction with ribs on the duct and an output parameter, Entropy Augmentation Generation Number (EAGN) is analysed when having an Al2O3 nanofluid (ϕ=3%) as working medium. In furtherance, tapes of isosceles triangular projections with variable configuration, such as rib angles (300<α< 900), Internal angle (300<β<900) and projection distance (1mm< x< 5mm) were also inserted in place of plain twisted tape inserts to verify the enhancement promised by this alteration. As expected, rate of heat transfer due to the presence of isosceles triangle projections gave decent augmentation. Later Response Surface based optimization was employed with non-parametric regression and genetic algorithm to make an investigative search of all the modified parameters so as to suggest best blend of inputs for low Entropy Augmentation Generation Number. Optimum performance was obtained at rib angle of 300, projection distance of 2.1mm and Internal angle of 44.40 with entropy augmentation number value of 0.77. The performance of Genetic Algorithm was compared with Micro Genetic Algorithm; it shows that optimized result is obtained less than half the time using Micro Genetic Algorithm.

Prediction of Sewage Treatment Cost in Rural Regions with Multivariate Adaptive Regression Splines

In this paper, to interpret the cost structure of decentralized wastewater treatment plants (DWWTPs) in rural regions, a simple nonparametric regression algorithm known as multivariate adaptive regression spline (MARS) was proposed and applied to simulate the construction cost (CC), operation and maintenance cost (OMC), and total cost (TC). The effects of design treatment capacity (DTC), removal efficiency of chemical oxygen demand (RCOD), and removal efficiency of ammonia nitrogen (RNH3-N) on the cost functions of CC, OMC, and TC were analyzed in detail. The results indicated that: (1) DTC is the most important parameter to determine cost structure with relative importance of 100%, followed by RCOD and RNH3-N with relative importance of 16.55%, and 9.75%, respectively; (2) when DTC is less than 5 m3/d, the slopes of CC and TC on DTC are constants of 1.923 and 1.809, respectively, with no relationship with RCOD and RNH3-N; (3) when DTC is less than 20 m3/d, the OMC is a constant of 435 RMB/year; and (4) in other cases, CC, OMC, and TC are related to RCOD and RNH3-N besides DTC. Compared with widely used support vector machine (SVM) models and multiple linear regression (MLR) models, the MARS model has better statistical significance with greater R values and smaller RMSE and MAPE values, which indicated that the MARS model is a better way to approximate the cost for DWWTPs.

Coupling Multivariate Adaptive Regression Spline (MARS) and Random Forest (RF)

In this article, a new algorithm to select the relevant features is proposed for handling microarray data with the specific aim of increasing classification accuracy. In particular, the optimal genes are extracted using filter and wrapper feature selection algorithms. Here, the use of non-parametric regression algorithm called Multivariate Adaptive Regression Spline (MARS) followed by proposed Random Forest Statistical Test (RFST) algorithm are being studied. The study evaluates the comparative performance of the results of RFST and MARS with existing algorithms on ten standard microarray datasets. For performance analysis, three parameters are taken into consideration, namely, the number of selected features, runtime, and classification accuracy. Experimental results indicate that different feature selection algorithms yield different candidate gene subset; therefore, a Hybrid approach is applied to determine the best candidate genes to provide maximum information about the disease. The findings foretell that the RFST is performing better on six out of ten datasets whereas MARS is performing better on other datasets.

Bootstrap Aggregating Multivariate Adaptive Regression Splines (Bagging MARS) to Analyse the Lecturer Research Performance in Private University

This study explores the use of a fairly simple nonparametric regression algorithm known as multivariate adaptive regression splines (MARS) which has the ability to approximate the relationship between the predictor and response variables. The main advantages of MARS are its capacity to produce simple, easy-to-interpret models, its ability to estimate the contributions of the predictor variables, and its computational efficiency. In this research, MARS is combined with one of nonparametric approach bootstrap aggregating (bagging). Bagging is used to improve the classification accuracy of the MARS method. This study is aimed to analyse lecturer research performance in private university using Bagging MARS algorithm. In modelling bagging MARS for lecturer research performance in a private university that there are three dominant influencing factors: 1) amount of research with cost an internal college granted by internal college with interest level of 86.20%; 2) number of publications of research results in national journals with interest level of 69.83%; and 3) number of speakers within national scientific meeting/ seminar with interest level of 63.34%. The accuracy of the MARS model classification is 84.615% with the classification error rate (APER) of 15.385%.

Quantifying Vegetation Biophysical Variables from Imaging Spectroscopy Data: A Review on Retrieval Methods.

An unprecedented spectroscopic data stream will soon become available with forthcoming Earth-observing satellite missions equipped with imaging spectroradiometers. This data stream will open up a vast array of opportunities to quantify a diversity of biochemical and structural vegetation properties. The processing requirements for such large data streams require reliable retrieval techniques enabling the spatiotemporally explicit quantification of biophysical variables. With the aim of preparing for this new era of Earth observation, this review summarizes the state-of-the-art retrieval methods that have been applied in experimental imaging spectroscopy studies inferring all kinds of vegetation biophysical variables. Identified retrieval methods are categorized into: (1) parametric regression, including vegetation indices, shape indices and spectral transformations; (2) nonparametric regression, including linear and nonlinear machine learning regression algorithms; (3) physically based, including inversion of radiative transfer models (RTMs) using numerical optimization and look-up table approaches; and (4) hybrid regression methods, which combine RTM simulations with machine learning regression methods. For each of these categories, an overview of widely applied methods with application to mapping vegetation properties is given. In view of processing imaging spectroscopy data, a critical aspect involves the challenge of dealing with spectral multicollinearity. The ability to provide robust estimates, retrieval uncertainties and acceptable retrieval processing speed are other important aspects in view of operational processing. Recommendations towards new-generation spectroscopy-based processing chains for operational production of biophysical variables are given.

Deciding on the support schemes for upcoming wind farms in competitive electricity markets

A variety of policies have driven escalated global growth of wind-power generation in recent years. However, wind production should be supported through market-based schemes that avoid overcompensation. The present paper aims to determine a justified support scheme for upcoming wind farms in competitive electricity markets. Clustering tools and non-parametric regression algorithms are employed to model the price-formation process in the market and estimate the revenue of forthcoming wind facilities. Accordingly, the premium paid to an upcoming wind farm is calculated by incorporating its estimated revenue and levelized cost of energy. Then, the impact of the proposed wind project on wholesale and retail electricity prices is modelled based on the achieved non-parametric regression models. The methodology is applied to two wind farms in the Alberta and Ontario electricity markets in Canada as the case studies. Our results indicate that electricity consumers in Ontario should pay a higher premium for each unit of energy generated by the wind farm due to its lower revenue from the market. However, it is observed in both markets that the impact of wind farms on wholesale prices exceed the remuneration paid to them and thus, electricity consumers experience a decrease in their ultimate electricity costs.

A Short-Term Traffic Flow Forecasting Method Based on a Three-Layer K-Nearest Neighbor Non-Parametric Regression Algorithm

Short-term traffic flow is one of the core technologies to realize traffic flow guidance. In this article, in view of the characteristics that the traffic flow changes repeatedly, a short-term traffic flow forecasting method based on a three-layer K-nearest neighbor non-parametric regression algorithm is proposed. Specifically, two screening layers based on shape similarity were introduced in K-nearest neighbor non-parametric regression method, and the forecasting results were output using the weighted averaging on the reciprocal values of the shape similarity distances and the most-similar-point distance adjustment method. According to the experimental results, the proposed algorithm has improved the predictive ability of the traditional K-nearest neighbor non-parametric regression method, and greatly enhanced the accuracy and real-time performance of short-term traffic flow forecasting.

Nonlinear structural modeling using multivariate adaptive regression splines

Various computational tools are available for modeling highly nonlinear structural engineering problems that lack a precise analytical theory or understanding of the phenomena involved. This paper adopts a fairly simple nonparametric adaptive regression algorithm known as multivariate adaptive regression splines (MARS) to model the nonlinear interactions between variables. The MARS method makes no specific assumptions about the underlying functional relationship between the input variables and the response. Details of MARS methodology and its associated procedures are introduced first, followed by a number of examples including three practical structural engineering problems. These examples indicate that accuracy of the MARS prediction approach. Additionally, MARS is able to assess the relative importance of the designed variables. As MARS explicitly defines the intervals for the input variables, the model enables engineers to have an insight and understanding of where significant changes in the data may occur. An example is also presented to demonstrate how the MARS developed model can be used to carry out structural reliability analysis.

Multivariate adaptive regression splines and neural network models for prediction of pile drivability

Piles are long, slender structural elements used to transfer the loads from the superstructure through weak strata onto stiffer soils or rocks. For driven piles, the impact of the piling hammer induces compression and tension stresses in the piles. Hence, an important design consideration is to check that the strength of the pile is sufficient to resist the stresses caused by the impact of the pile hammer. Due to its complexity, pile drivability lacks a precise analytical solution with regard to the phenomena involved. In situations where measured data or numerical hypothetical results are available, neural networks stand out in mapping the nonlinear interactions and relationships between the system's predictors and dependent responses. In addition, unlike most computational tools, no mathematical relationship assumption between the dependent and independent variables has to be made. Nevertheless, neural networks have been criticized for their long trial-and-error training process since the optimal configuration is not known a priori. This paper investigates the use of a fairly simple nonparametric regression algorithm known as multivariate adaptive regression splines (MARS), as an alternative to neural networks, to approximate the relationship between the inputs and dependent response, and to mathematically interpret the relationship between the various parameters. In this paper, the Back propagation neural network (BPNN) and MARS models are developed for assessing pile drivability in relation to the prediction of the Maximum compressive stresses (MCS), Maximum tensile stresses (MTS), and Blow per foot (BPF). A database of more than four thousand piles is utilized for model development and comparative performance between BPNN and MARS predictions.

Multivariate adaptive regression splines model for reliability assessment of serviceability limit state of twin caverns

Construction of a new cavern close to an existing cavern will result in a modification of the state of stresses in a zone around the existing cavern as interaction between the twin caverns takes place. Extensive plane strain finite difference analyses were carried out to examine the deformations induced by excavation of underground twin caverns. From the numerical results, a fairly simple nonparametric regression algorithm known as multivariate adaptive regression splines (MARS) has been used to relate the maximum key point displacement and the percent strain to various parameters including the rock quality, the cavern geometry and the in situ stress. Probabilistic assessments on the serviceability limit state of twin caverns can be performed using the First-order reliability spreadsheet method (FORM) based on the built MARS model. Parametric studies indicate that the probability of failure Pf increases as the coefficient of variation of Q increases, and Pf decreases with the widening of the pillar.