Weighted Support Vector Regression Research Articles

Semiconductor Price Index Predicting Based on a Novel Improved AdaBoost Feature-Weighted Combination Model

The semiconductor price index serves as a vital metric for assessing technological developments and related market trends. Establishing a more accurate forecasting model for the semiconductor price index is of significant importance for analyzing the industry’s trends and market directions. In this paper, a novel framework for semiconductor price index forecasting is proposed. In addition to traditional financial data, the study introduces search engine data (Google Trends) representing investor attention, and introduces text information extracted from online news headlines reflecting major market events and government policies as independent variables. Used to predict the dependent variable: The PHLX Semiconductor Sector (SOX). First, the XGBoost model is employed to compute the importance scores of each feature. Then, a feature weight coefficient indicator is constructed based on these importance scores to calculate the weight coefficient indicator values for each feature. These indicator values are then used to weight the kernel function of Support Vector Regression (SVR), resulting in weighted Support Vector Regression (WSVR). Finally, WSVR is utilized as the base learner for Adaptive Boosting (AdaBoost), yielding the XGBoost–WSVR–AdaBoost model based on feature weighting. The proposed model outperforms AdaBoost, RNN, ERT, LSTM, and other models in terms of Mean Absolute Percentage Error (MAPE) and goodness-of-fit (R2\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$${R}^{2}$$\\end{document}). It also exhibits superior predictive performance compared to models in ablation experiments, and the introduction of text data or Google trends further improves the prediction performance of the model. In conclusion, the improved AdaBoost feature-weighted combination model proposed in this study offers a more accurate prediction for semiconductor price index.

Adaptive locally weighted support vector algorithm with asymmetrically parametric insensitive/margin model

In many fields such as finance and electrical load, locally weighted support vector regression (LWSVR) method can achieve better prediction results by constructing different sub-models for each prediction target. However, it selected a fixed and single nearest neighbor parameter of kglobal for each query, and adopted the traditional k-nearest neighbor (KNN) algorithm, which is suboptimal. The weights in LWSVR assigned to the neighboring sample points also affected the accuracy of the model predictions. In this paper, we propose an adaptive locally weighted support vector regression with asymmetrically parametric insensitive/margin model, called Asy-par-v-ALWSVR. Since the selection of similar sample for query points is significant. We propose a novel locally q-neighbour selection algorithm that adaptively selects the most suitable number of neighbours for each query point based on Mahalanobis distance, rather than using a fixed kglobal. This significantly improves prediction accuracy. The weights of neighbours for each query point in Asy-par-v-ALWSVR is based on the distance within the kernel-induced feature space. This effectively discerns the importance of samples during the training process. In particular, Asy-par-v-ALWSVR can handle skewed noise with heteroscedastic structure in regression problems. Finally, we evaluate the proposed model on a synthetic sinc dataset and 14 real datasets. The results demonstrate that Asy-par-v-ALWSVR outperforms six state-of-the-art SVR-based models.

A hybrid framework for short term load forecasting with a navel feature engineering and adaptive grasshopper optimization in smart grid

Short-term load forecasting (STLF) enables distribution system operators to perform efficient energy management by flexibly engaging energy consumers under the intelligent demand-response program in the smart grid (SG). This paper develops a fast and accurate hybrid load forecasting model with this motivation. The proposed model integrates a locally weighted support vector regression (LWSVR) based forecaster with two modules. These modules are feature engineering (FE) and adaptive grasshopper optimization (AGO) based optimizers. In the FE module, firstly, the proposed hybrid feature selector (HFS) is developed using wrapper and filter techniques to determine an optimal subset of features. Furthermore, the instance-based Relief-F (REF) and information theoretic-based mutual information (MI) filters are used to decrease the curse of feature dimensionality by finding and eradicating extraneous features. In addition, to overcome the overfitting problem, the HFS module is further optimized using the recursive feature elimination (RFE) wrapper feature selection technique. The essential features are then extracted using a radial basis Kernel-based principal component analysis (RBF-KPCA) algorithm to eliminate the dimensionality reduction problem. The AGO algorithm tunes the LWSVR model’s appropriate parameters to effectively evade entrapping into local optimum and yield accurate prediction results. However, the efficacy and productiveness of the forecasting model are differentiated equally by its convergence rate and stability. Actual hourly load data of two states of Australia (New South Wales (NSW) and Victoria (VIC)) and California Independent System Operator United States (CAISO-US) are employed as a case study to estimate the effectiveness and applicability of the designed model. Empirical results show that the devised model surpasses benchmark models (single and hybrid) in terms of stability, accuracy, and convergence rate.

Stock price prediction using multi-scale nonlinear ensemble of deep learning and evolutionary weighted support vector regression

Abstract Stock price prediction has become a focal topic for relevant investors and scholars in these years. However, owning to the non-stationarity and complexity of stock price data, it is challenging to predict stock price accurately. This research develops a novel multi-scale nonlinear ensemble learning framework for stock price prediction, which consists of variational mode decomposition (VMD), evolutionary weighted support vector regression (EWSVR) and long short-term memory network (LSTM). The VMD is utilized to extract the basic features from an original stock price signal and eliminate the disturbance of illusive components. The EWSVR is utilized to predict each sub-signal with corresponding features, whose penalty weights are determined according to the time order and whose parameters are optimized by tree-structured Parzen estimator (TPE). The LSTM-based nonlinear ensemble learning paradigm is employed to integrate the predicted value of each sub-signal into the final prediction result of stock price. Four real prediction cases are utilized to test the proposed model. The proposed model’s prediction results of multiple evaluation metrics are significantly improved compared to other benchmark models both in stock market closing price forecasting.

Intelligent computing and analysis of breakdown voltage of rod-rod long air gap

The discharge characteristics of air gap is one of the important factors to determine the insulation level which is crucial for electrical engineering construction. In order to calculate the 50% breakdown voltage of rod-rod air gap under different meteorological conditions and different gap distances, we take the positive standard switching impulse test data of rod-rod air gap under different meteorological conditions as samples. Based on training set, a breakdown voltage prediction model of rod-rod air gap under different meteorological conditions was established by cuckoo search optimized weighted support vector regression (CS-ω-SVR). The results show that the average absolute percentage error of the test set is 3.6%, which verifies the validity of this computing method. The method can provide reference for the computation of air gap breakdown voltage under different altitudes and extreme meteorological conditions.

A New L₁ Multi-Kernel Learning Support Vector Regression Ensemble Algorithm With AdaBoost

This paper proposes a new multi-kernel learning ensemble algorithm, called Ada-<inline-formula> <tex-math notation="LaTeX">$L_{1}$ </tex-math></inline-formula>MKL-WSVR, which can be regarded as an extension of multi-kernel learning (MKL) and weighted support vector regression (WSVR). The first novelty is to add the <inline-formula> <tex-math notation="LaTeX">$L_{1}$ </tex-math></inline-formula> norm of the weights of the combined kernel function to the objective function of WSVR, which is used to adaptively select the optimal base models and their parameters. In addition, an accelerated method based on fast iterative shrinkage thresholding algorithm (FISTA) is developed to solve the weights of the combined kernel function. The second novelty is to propose an integrated learning framework based on AdaBoost, named Ada-<inline-formula> <tex-math notation="LaTeX">$L_{1}$ </tex-math></inline-formula>MKL-WSVR. In this framework, we integrate FISTA into AdaBoost. At each iteration, we optimize the weights of the combined kernel function and update the weights of the training samples at the same time. Then an ensemble regression function of a set of regression functions is output. Finally, two groups of the experiments are designed to verify the performance of our algorithm. On the first group of the experiments including eight datasets from UCI machine learning repository, the MAEs and RMSEs of Ada-<inline-formula> <tex-math notation="LaTeX">$L_{1}$ </tex-math></inline-formula>MKL-WSVR are reduced by 11.14&#x0025; and 9.08&#x0025; on average, respectively. Furthermore, on the second group of the experiments including the COVID-19 epidemic datasets from eight countries, the MAEs and RMSEs of Ada-<inline-formula> <tex-math notation="LaTeX">$L_{1}$ </tex-math></inline-formula>MKL-WSVR are reduced by 31.19&#x0025; and 29.98&#x0025; on average, respectively.

A robust prediction model for displacement of concrete dams subjected to irregular water‐level fluctuations

AbstractMonitoring and predicting the displacement of concrete dams is one of the most crucial considerations for ensuring their long‐term safe operation. Most existing models are designed for dams subjected to common structural and environmental conditions, with little attention paid to atypical operational conditions, such as structural strengthening or sudden changes in the external environment. The motivation for this work is to develop a reliable prediction model for displacement behavior of concrete dams subjected to irregular upstream water‐level fluctuations. In our study, a reconstruction method for unevenly sampled time series is presented to analyze such data. Then, an improved model, factor weighted support vector regression (FWSVR), which differentiates various factors and their effects through a weighting matrix, is theoretically derived. The weights are determined by the RReliefF algorithm, and together with FWSVR form the final RReliefF‐based FWSVR (RFWSVR) model. In particular, the hyperparameters involved in the above modeling strategy are optimized by the Grey Wolf Optimizer. Eventually, the prediction robustness of the developed model was verified on the data from four representative monitoring points of a real‐world dam, where its accuracy was compared to classical dam behavior modeling methods, FWSVR models using other weighting methods, and an ensemble learning algorithm. Comparative evaluation of the performance of the different methods was conducted with the help of recognized statistical indices. The evaluation results show that the overall performance of the proposed RFWSVR model is optimal for the displacement prediction at the selected points when the dam case is subjected to irregular water‐level changes. This novel modeling approach may be generalized for modeling the evolution behavior of other civil or hydraulic structures.

Short term electric load forecasting using hybrid algorithm for smart cities

Many day-to-day operation decisions in a smart city need short term load forecasting (STLF) of its customers. STLF is a challenging task because the forecasting accuracy is affected by external factors whose relationships are usually complex and nonlinear. In this paper, a novel hybrid forecasting algorithm is proposed. The proposed hybrid forecasting method is based on locally weighted support vector regression (LWSVR) and the modified grasshopper optimization algorithm (MGOA). Obtaining the appropriate values of LWSVR parameters is vital to achieving satisfactory forecasting accuracy. Therefore, the MGOA is proposed in this paper to optimally select the LWSVR’s parameters. The proposed MGOA can be derived by presenting two modifications on the conventional GOA in which the chaotic initialization and the sigmoid decreasing criterion are employed to treat the drawbacks of the conventional GOA. Then the hybrid LWSVR-MGOA method is used to solve the STLF problem. The performance of the proposed LWSVR-MGOA method is assessed using six different real-world datasets. The results reveal that the proposed forecasting method gives a much better forecasting performance in comparison with some published forecasting methods in all cases.

An ensemble model based on weighted support vector regression and its application in annealing heating process

An ensemble model based on weighted support vector regression and its application in annealing heating process

Augmenting limited background monitoring data for improved performance in land use regression modelling: Using support vector regression and mobile monitoring

Land Use Regression (LUR) models was developed using fixed site and mobile station data in Dublin Ireland. The uncertainty associated with short term, mobile station data was accounted for using a weighted function, considering the differences in record duration of data across the study. A systematic buffer distance decay curve-based approach was also considered in this study to identify appropriate predictors and their associated buffer distance. The analysis was performed in Dublin city for fine particulate matter (PM2.5) where 10 fixed monitoring stations data were available, with average daily concentration data records of 1–7 years in length. 105 mobile station data were also collected at several locations in the city for durations of 12 h to 3 days. The performance of a developed multiple linear regression based LUR model was evaluated by using the leave-one-out-cross-validation procedure. The model explained variances (R2) of the LUR model were found to be 0.219 for winter and 0.688 for summer months. One of the primary reasons for poor performance of the LUR model was that the relationship between the predictors and the mean PM2.5 concentration were found to be non-linear which cannot be modelled by using multiple linear regression approach. To address this, a weighted support vector regression (WSVR) approach was also considered that can account for non-linearity while developing the LUR model. The model performance in terms of R2 was found to improve to 0.912 for winter and 0.916 for summer using this method. The developed WSVR model can be used to predict PM2.5 concentration at any ungauged locations in the study area with considerable accuracy, which can be the basis for future epidemiological studies.

Air Pollution Forecasting Model Based on Chance Theory and Intelligent Techniques

A novel approach of weighted support vector regression (WSVR) technique with applied chance theory was proposed to build a robust forecasting model, called the chance weighted support vector regression (chWSVR) model. In order to forecast the particulate matter air pollutant of diameter less than 10 micrometers (PM10) one hour advance in the Greater Cairo Metropolitan Area (GCMA) in Egypt. The chance theory has advanced concepts pertinent to treat cases where both randomness and fuzziness play simultaneous roles at one time. The basic idea based on the proposed chWSVR model is assigning the chance weight value of the target variable, based on the chance theory, to its corresponding dataset point to become minimized in the objective function making that point more significant during the training process. Measuring data were collected and reprocessed from four monitoring stations located in GCMA and relative to the springs during the period from 2007 to 2010. The results of such model compared to similar ones built by other machine learning techniques, Random Forest and Bootstrap aggregating techniques. In all stations, comparing such models revealed that the proposed chWSVR model findings were promising in the forecasting of PM10hourly concentration.

Reverse adaptive krill herd locally weighted support vector regression for forecasting and trading exchange traded funds

This study introduces a Reverse Adaptive Krill Herd-Locally Weighted Support Vector Regression (RKH-LSVR) model. The Reverse Adaptive Krill Herd (RKH) algorithm is a novel metaheuristic optimization technique inspired by the behavior of krill herds. In RKH-LSVR, the RKH optimizes the locally weighted Support Vector Regression (LSVR) parameters by balancing the search between local and global optima. The proposed model is applied to the task of forecasting and trading six ETF stocks on a daily basis over the period 2010–2015. The RKH-LSVR's efficiency is benchmarked against a set of traditional SVR structures and simple linear and non-linear models. The trading application is designed in order to validate the robustness of the algorithm under study and to provide empirical evidence in favor of or against the Adaptive Market Hypothesis (AMH). In terms of the results, the RKH-LSVR outperforms its counterparts in terms of statistical accuracy and trading efficiency, while the time varying trading performance of the models under study validates the AMH theory.

A new local pressure loss coefficient model of a duct tee junction applied during transient simulation of a HVAC air-side system

Most of the local pressure loss coefficient (LPLC) models for duct fittings used in heating, ventilation and air conditioning (HVAC) air-side system transient simulations are simplified. The LPLCs are defined as having a constant value at the rated flow condition or as having a polynomial function of the flow ratio (or velocity ratio). To determine the influence of these simplifications, this study used a diverging tee junction as an example. First, we performed CFD calculations to generate a new LPLC dataset and trained a data-driven model using feature weighted support vector regression (FWSVR) combined with particle swarm optimization (PSO-FWSVR). Finally, we compared this new LPLC model with the two traditional models at the level of both individual junction and air-side system. The results show that the accuracy of the new model is greatly improved and the LPLC model can have a significant impact on the system operation condition.

European Exchange Trading Funds Trading with Locally Weighted Support Vector Regression

In this paper, two different Locally Weighted Support Vector Regression (wSVR) algorithms are generated and applied to the task of forecasting and trading five European Exchange Traded Funds. The trading application covers the recent European Monetary Union debt crisis. The performance of the proposed models is benchmarked against traditional Support Vector Regression (SVR) models. The Radial Basis Function, the Wavelet and the Mahalanobis kernel are explored and tested as SVR kernels. Finally, a novel statistical SVR input selection procedure is introduced based on a principal component analysis and the Hansen, Lunde, and Nason (2011) model confidence test. The results demonstrate the superiority of the wSVR models over the traditional SVRs and of the v-SVR over the ε-SVR algorithms. We note that the performance of all models varies and considerably deteriorates in the peak of the debt crisis. In terms of the kernels, our results do not confirm the belief that the Radial Basis Function is the optimum choice for financial series.

Time series forecasting for building energy consumption using weighted Support Vector Regression with differential evolution optimization technique

Electricity load forecasting is crucial for effective operation and management of buildings. Support Vector Regression (SVR) have been successfully used in solving nonlinear regression and time series problems related to building energy consumption forecasting. As the performance of SVR heavily depends on the selection of its parameters, differential evolution (DE) algorithm is employed in this study to solve this problem. The forecasting model is developed using weighted SVR models with nu-SVR and epsilon-SVR. The DE algorithm is again used to determine the weights corresponding to each model. A case of time series energy consumption data from an institutional building in Singapore is used to elucidate the performance of the proposed model. The proposed model can be used to forecast both, half-hourly and daily electricity consumption time series data for the same building. The results show that for half-hourly data, the model exhibits higher weight for nu-SVR, whereas for daily data, a higher weight for epsilon-SVR is observed. The mean absolute percentage error (MAPE) for daily energy consumption data is 5.843 and that for half-hourly energy consumption is 3.767 respectively. A detailed comparison with other evolutionary algorithms show that the proposed model yields higher accuracy for building energy consumption forecasting.

One norm linear programming support vector regression

In this paper, a new linear programming formulation of a 1-norm support vector regression (SVR) is proposed whose solution is obtained by solving an exterior penalty problem in the dual space as an unconstrained minimization problem using Newton method. The solution of modified unconstrained minimization problem reduces to solving just system of linear equations as opposed to solving quadratic programming problem in SVR, which leads to extremely simple and fast algorithm. The algorithm converges from any starting point and can be easily implemented in MATLAB without using any optimization packages. The main advantage of the proposed approach is that it leads to a robust and sparse model representation meaning that many components of the optimal solution vector will become zero and therefore the decision function can be determined using much less number of support vectors in comparison to SVR, smooth SVR (SSVR) and weighted SVR (WSVR). To demonstrate its effectiveness, experiments were performed on well-known synthetic and real-world benchmark datasets. Similar or better generalization performance of the proposed method in less training time in comparison with SVR, SSVR and WSVR clearly exhibits its suitability and applicability.

Least squares support vector regression and interval type-2 fuzzy density weight for scene denoising

Support vector machines are the popular machine learning techniques. Its variant least squares support vector regression (LS-SVR) is effective for image denoising. However, the fitting of the samples contaminated by noises in the training phase will result in the fact that LS-SVR cannot work well when noise level is too far from it or noise density is high. Type-2 fuzzy sets and systems have been shown to be a more promising method to manifest the uncertainties. Various noises would be taken as uncertainties in scene images. By integrating the design of learning weights with type-2 fuzzy sets, a systematic design methodology of interval type-2 fuzzy density weighted support vector regression (IT2FDW-SVR) model for scene denoising is presented to address the problem of sample uncertainty in scene images. A novel strategy is used to design the learning weights, which is similar to the selection of human experience. To handle the uncertainty of sample density, interval type-2 fuzzy logic system (IT2FLS) is employed to deduce the fuzzy learning weights (IT2FDW) in the IT2FDW-SVR, which is an extension of the previously weighted SVR. Extensive experimental results demonstrate that the proposed method can achieve better performances in terms of both objective and subjective evaluations than those state-of-the-art denoising techniques.

On Weighted Support Vector Regression

We propose a new type of weighted support vector regression (SVR), motivated by modeling local dependencies in time and space in prediction of house prices. The classic weights of the weighted SVR are added to the slack variables in the objective function (OF‐weights). This procedure directly shrinks the coefficient of each observation in the estimated functions; thus, it is widely used for minimizing influence of outliers. We propose to additionally add weights to the slack variables in the constraints (CF‐weights) and call the combination of weights the doubly weighted SVR. We illustrate the differences and similarities of the two types of weights by demonstrating the connection between the Least Absolute Shrinkage and Selection Operator (LASSO) and the SVR. We show that an SVR problem can be transformed to a LASSO problem plus a linear constraint and a box constraint. We demonstrate the capabilities of the doubly weighted approach through an example of prediction of house prices. The weight functions in the house pricing model depend on the geographical distance to the house of interest and the difference in time of sale (CF‐weights) as well as the differences lying in variables (OF‐weights), such as house size and number of floors. The results illustrate that the combination of the two types of weights describes the relative importance of observations very well and lowers the influence of possible outliers. Therefore, it enables the SVR models to have good performance. Copyright © 2014 John Wiley &amp; Sons, Ltd.

An Asymmetric Weighted SVR for Construction Final Accounting Prediction

Asymmetric weighted support vector regression not only can further lower computational complexity by transforming a quadratic programming problem into a linear equation group problem, but also has the different penalty parameters to obtain robust regression results. Thus, an asymmetric weighted support vector regression algorithm is presented and applied for construction final accounting prediction. The dimension of the input vector of the training sample need to be firstly determined before constructing the regression model, as the dimension of the input vector of the training sample has a great influence on the testing effects of the prediction models. Thus, the asymmetric weighted SVR models are trained by the 2 s 5 dimensional input vector respectively to find the optimal dimension number of the input vector of the training samples. The testing results show that the mean construction final account evaluation error of asymmetric weighted support vector regression is smaller than that of asymmetric weighted SVR. Therefore, asymmetric weighted support vector regression is more suitable for construction final accounting evaluation than least square SVR or traditional SVR model.

New Strategy for Analog Circuit Performance Evaluation under Disturbance and Fault Value

Focus on this issue of disturbance and fault value is inevitable in data collection about analog circuit. A novel strategy is developed for analog circuit online performance evaluation based on fuzzy learning and double weighted support vector machine (DWMK-FSVM). First, the double weighted support vector regression machine is employed to be the indirect evaluation means, relied on the college analog electronic technology experiment to evaluate analog circuit. Second, the superiority of fuzzy learning also is addressed to realize active suppression to the fault values and disturbance parameters. Moreover, the multikernel RBF is employed by support vector regression machine to realize more flexibility online such as the bandwidths tuning. Numerical results, supported by the college analog circuit experiments, adopted OTL performance eight indexes, which were obtained via precision instrument evaluation in two years to construct training set and are then to be evaluated online based on DWMK-FSVM. Simulation results presented not only highlight precision of the evaluation strategy derived here but also illustrate its great robustness.