Twin Support Vector Regression Research Articles

Foretelling the compressive strength of concrete using twin support vector regression

Foretelling the compressive strength of concrete using twin support vector regression

Nonparametric Modelling of Ship Dynamics Using Puma Optimizer Algorithm-Optimized Twin Support Vector Regression

Ship dynamic models serve as the foundation for designing ship controllers, trajectory planning, and obstacle avoidance. Support vector regression (SVR) is a commonly used nonparametric modelling method for ship dynamics. Achieving high accuracy SVR models requires a substantial amount of training samples. Additionally, as the number of training samples increases, the computational efficiency for solving the quadratic programming problem (QPP) of SVR decreases. Ship controllers demand dynamic models with both high accuracy and computational efficiency. Therefore, to enhance the prediction accuracy and computational efficiency of SVR, this paper proposes a nonparametric modelling method based on twin SVR (TSVR). TSVR replaces a large QPP with a set of smaller QPPs, significantly enhancing generalizability and computational efficiency. To further improve the predictive accuracy of TSVR, the puma optimizer algorithm is employed to determine the optimal hyperparameters. The performance of the proposed method is validated using a Mariner class vessel. Gaussian white noise is introduced into the modelling data to simulate measurement error. The TSVR model accurately predicts various zigzag and turning circle manoeuvring motions under disturbance conditions, demonstrating its robustness and generalizability. Compared to the SVR model, the TSVR model achieves lower root mean square error and computational time, confirming its superior predictive accuracy and computational efficiency.

Groundwater level prediction using an improved ELM model integrated with hybrid particle swarm optimisation and grey wolf optimisation

Groundwater, the world's most abundant source of freshwater, is rapidly depleting in many regions due to a variety of factors. Groundwater resources (GWR) play a crucial role in daily life, economic progress, and agricultural crop production, which makes groundwater level (GWL) prediction much more significant for enhanced management. However, one of the major challenges in investigating and estimating groundwater reduction and managing its resources is the absence of complete and reliable data. Therefore, the application of artificial intelligence (AI) models with the necessity for less data and better prediction accuracy is inevitable. The performances of hybrid models including the combination of particle swarm optimisation and grey wolf optimisation with extreme learning machine (ELM-PSOGWO) in estimating the GWL are investigated and evaluated against ELM-GWO, ELM-PSO, ELM, Twin-support vector regression (T-SVR) and SVR models based on the coefficient of determination (R2), mean absolute error (MAE), root mean square error (RMSE), and Nash-Sutclife efficiency coefficient (NSC). Outcomes of this study showed that applied hybridised ELM-PSOGWO (R2 = 0.9969, RMSE = 0.0088, NSC = 0.9961, MAE = 0.0047) performed best, followed by ELM-GWO, ELM-PSO, ELM, TSVR, and SVR in predicting GWL. Based on the obtained results, it is evident that the ELM-PSOGWO model illustrates an excellent agreement with observed GWL data. To this end, this work reveals that hybrid optimisation algorithms can enhance performance of ELM model significantly in GWL prediction. Utilization of advanced AI techniques can well estimate groundwater systems, which saves resources and labor, employed conventionally.

Twin proximal support vector regression with heteroscedastic Gaussian noise

Twin proximal support vector regression, a novel approach, merges twin support vector regression (TSVR) with proximal support vector regression (PSVR) for regression analysis. Classical PSVR and TSVR assume that the data noise follows a homoscedastic Gaussian distribution with zero mean. However, in the real world, such as short-term wind speed prediction and wind farm power generation prediction, noise tends to follow a heteroscedastic Gaussian distribution with zero mean. Therefore, based on the aforementioned model framework and added the heteroscedastic noise feature, we develop a novel regression model named the twin proximal support vector regression model with heteroscedastic Gaussian noise (TPSVR-HGN). The Augment Lagrange multiplier approach is used to solve the proposed TPSVR-HGN model. On the artificial dataset and the real world dataset, compared with recent advanced models, the prediction accuracy of the TPSVR-HGN model has been improved by about 7% and 11% respectively. As the results of this experiment indicate, our method is effective and feasible.

Multi-hyperplane twin support vector regression guided with fuzzy clustering

In recent years, twin support vector regression has become a hot research topic because of its low computing time and excellent performance. It can be observed, however, that either the support vector regression or twin support vector regression have no more than two regression hyperplanes. Many research studies have ignored the potential of multiple hyperplanes regression algorithms. In this paper, a one-versus-all twin support vector regression (OVATWSVR) with multiple regression hyperplanes is proposed, in order to achieve excellent regression performance though multi-hyperplane structure. Suppose that the input data implicitly has p categories, OVATWSVR solves a smaller quadratic programming problem (QPP) and repeats this process p times, resulting in p regression hyperplanes. For the purpose of mining the implicit category information of each point to assist OVATWSVR in training hyperplanes, meanwhile considering the fuzzy characteristics of points (they lack classification labels) in different types of regression datasets, we further propose a fuzzy clustering algorithm, namely fuzzy weighted K-nearest neighbors fuzzy density peak clustering (FKNN-FDPC), to provide OVATWSVR with information regarding the category of each point. A fuzzy membership function, also guided by FKNN-FDPC, is added to OVATWSVR in order to enhance the capability of OVATWSVR to handle possible fuzzy properties in data, thus creating FOVATWSVR. F3OVATWSVR is a reasonable name for the entire multiple phases’ algorithm. Several UCI benchmark datasets, a real-world competition dataset and a state of health (SOH) estimation of lithium-ion batteries dataset are used to verify the superiority and effectiveness of F3OVATWSVR.

Long Short-Term Memory-Based Twin Support Vector Regression for Probabilistic Load Forecasting.

A probabilistic load forecast that is accurate and reliable is crucial to not only the efficient operation of power systems but also to the efficient use of energy resources. In order to estimate the uncertainties in forecasting models and nonstationary electric load data, this study proposes a probabilistic load forecasting model, namely BFEEMD-LSTM-TWSVRSOA. This model consists of a data filtering method named fast ensemble empirical model decomposition (FEEMD) method, a twin support vector regression (TWSVR) whose features are extracted by deep learning-based long short-term memory (LSTM) networks, and parameters optimized by seeker optimization algorithms (SOAs). We compared the probabilistic forecasting performance of the BFEEMD-LSTM-TWSVRSOA and its point forecasting version with different machine learning and deep learning algorithms on Global Energy Forecasting Competition 2014 (GEFCom2014). The most representative month data of each season, totally four monthly data, collected from the one-year data in GEFCom2014, forming four datasets. Several bootstrap methods are compared in order to determine the best prediction intervals (PIs) for the proposed model. Various forecasting step sizes are also taken into consideration in order to obtain the best satisfactory point forecasting results. Experimental results on these four datasets indicate that the wild bootstrap method and 24-h step size are the best bootstrap method and forecasting step size for the proposed model. The proposed model achieves averaged 46%, 11%, 36%, and 44% better than suboptimal model on these four datasets with respect to point forecasting, and achieves averaged 53%, 48%, 46%, and 51% better than suboptimal model on these four datasets with respect to probabilistic forecasting.

Robust twin support vector regression with correntropy-based metric

Robust twin support vector regression with correntropy-based metric

Enhanced machine learning model via twin support vector regression for streamflow time series forecasting of hydropower reservoir

The non-stationary, complex, and non-linear characteristics of streamflow time series have a significant impact on the simulation results of the conventional hydrological forecasting models. To improve the performances, this paper develops an enhanced machine learning model for streamflow time series forecasting, where the twin support vector regression (TSVR) is combined with singular spectrum analysis (SSA) and grey wolf optimizer (GWO). Specially, the SSA method is set as the data preprocessing tool for pattern identification; the TSVR model is set as the basic forecasting module for each pattern and the GWO method is used as the optimizer to select feasible parameter combination. Multi-step-ahead streamflow forecasting tasks are used to examine the feasibility and predictability of the proposed model. The results indicate that the proposed model can yield superior results compared with the traditional forecasting models. Thus, a robust and reliable tool is provided for streamflow time series forecasting under uncertainty.

Degradation trend evaluation for smart meters under high dry heat natural environments

Smart Meters (SMs) degradation trend evaluation (DTE) is critical for accurate electricity metering and improved energy efficiency, particularly in extreme natural environments. However, environmental noise and insufficient interpretability often limit the performance of DTE. To remedy this problem, an optimized local density-based (OLD) method is first proposed for outlier identification, where a modified distance measurement and double nearest neighbors are used to improve the identification performance. Next, a DTE model, namely multi-kernel twin support vector regression (MTSVR), is presented to combine multiple environmental features using the proposed up- and down-kernel combination method. Integrating the MTSVR and OLD, real-world SMs datasets collected from a high dry heat area in China are utilized for model verification. The results demonstrate that the OLD-MTSVR framework has a superior performance for SMs degradation trend analysis. More importantly, the feature diversity can be analyzed quantitatively through the model interpretability.

A Novel Twin Support Vector Regression Model for Wind Speed Time-Series Interval Prediction

Although the machine-learning model demonstrates high accuracy in wind speed prediction, it struggles to accurately depict the fluctuation range of the predicted values due to the inherent uncertainty in wind speed sequences. To address this limitation and enhance the reliability, we propose an effective wind speed interval prediction model that combines twin support vector regression (TSVR), variational mode decomposition (VMD), and the slime mould algorithm (SMA). In our methodology, the complex wind speed series is decomposed into multiple relatively stable subsequences using the VMD method. The principal component and residual series are then subject to interval prediction using the TSVR model, while the remaining components undergo point prediction. The SMA method is employed to search for optimal parameter combinations. The prediction interval of wind speed is obtained by aggregating the forecasting results of all TSVR models for each subseries. Our proposed model has demonstrated superior performance in various applications. It ensures that the wind speed value falls within the designated interval range while achieving the narrowest prediction interval. For instance, in the spring dataset with 1-period, we obtained a predicted interval with a prediction intervals coverage probability (PICP) value of 0.9791 and prediction interval normalized range width (PINRW) value of 0.0641. This outperforms other comparative models and significantly enhances its practical application value. After adding the residual interval prediction model, the reliability of the prediction interval is significantly improved. As a result, this study presents a novel twin support vector regression model as a valuable approach for multi-step wind speed interval prediction.

Deriving hydropower reservoir operation policy using data-driven artificial intelligence model based on pattern recognition and metaheuristic optimizer

Robust reservoir operation policies are crucial in ensuring the effective utilization of water resources. However, owing to multiple complicated and changeable factors in practice, it is difficult for standalone approaches to derive reasonable reservoir operation policy. To address the practical requirement, this research proposes a novel artificial intelligence method for deriving reservoir operation policy. The proposed method uses the fuzzy clustering iteration method to identify multiple typical operation patterns from the influencing factors; secondary, for all the samples within each pattern, the novel twin support vector regression (TSVR) is utilized to model the nonlinear mapping relationship between the influence inputs and the target outputs, while the emerging equilibrium optimizer is chosen to determine suitable computation parameters for the TSVR model. The feasibility of the proposed method is fully evaluated on two real-world huge hydropower reservoirs in China. The simulations demonstrate that the developed method can yield better comprehensive benefits than several control methods in deriving reservoir operation policy under uncertain environments. Hence, the experiments confirm that metaheuristic algorithms and pattern recognition techniques can enhance the performance of a standalone artificial intelligence methods in deriving reservoir operation policy.

Mode decomposition based large margin distribution machines for sediment load prediction

Precise suspended sediment load (SSL) prediction is essential for irrigation, hydropower and river management practices. But due to some external factors such as high altitude, heavy monsoon and tropical climate conditions, the data we collect may contain noisy samples. Hence, it becomes a challenging task to accurately predict the SSL in rivers. Therefore, to diminish the influence of noise, the empirical mode decomposition (EMD) -based techniques can be adopted. Moreover, the large margin distribution machine-based regression (LDMR) can deal efficiently with noisy datasets as it simultaneously minimizes the insensitive loss as well as quadratic loss. The significance of this work lies in its contribution to improving the accuracy of suspended sediment load (SSL) prediction in rivers, which has practical implications for various applications such as irrigation, hydropower, and river management. We recognize the challenges posed by external factors, which introduce noise into the collected data, making accurate prediction of SSL difficult. Overall, the significance of this work lies in its novel integration of EMD-based techniques and LDMR-based models, which address the challenges posed by noisy and non-stationary sediment load data. The findings contribute to the field of SSL prediction, offering practical solutions for managing and utilizing river resources more effectively in various domains. The main advantage of the least squares LDMR (LS-LDMR) approach is that it solves a system of linear equations rather than solving a large QPPs unlike SVR, TSVR, LDMR, which makes it computationally efficient. It is well known that the sediment load data is complex and non-stationary in nature. Therefore, for daily SSL prediction, the LDMR and least squares LDMR (LSLDMR) models are embedded with two different decomposition techniques, EMD and ensemble EMD (EEMD), to handle the nonlinear and non-stationary characteristics of sediment load data on the forecasted outputs and to increase prediction ability. The results of the proposed EMD-LDMR, EEMD-LDMR, EMD-LSLDMR and EEMD-LSLDMR are compared with the conventional support vector regression (SVR), twin SVR (TSVR), LDMR and LSLDMR. The performance of the evaluated using the best-fit model using the mean absolute error (MAE), root mean square error (RMSE), symmetric mean absolute percentage error (SMAPE), Willmott’s index (WI), correlation coefficient (CC) and R2. Better or comparable results specify the efficiency of the suggested models. For graphical visualization, scatterplots, prediction error plots and violin plots are shown. Numerical results show that these hybrid models show excellent prediction performance. The overall analysis of the results suggests using EEMD-LSLDMR for SSL estimation.

Brain Age Prediction With Improved Least Squares Twin SVR.

Alzheimer's disease (AD) is the prevalent form of dementia and shares many aspects with the aging pattern of the abnormal brain. Machine learning models like support vector regression (SVR) based models have been successfully employed in the estimation of brain age. However, SVR is computationally inefficient than twin support vector machine based models. Hence, different twin support vector machine based models like twin SVR (TSVR), ε-TSVR, and Lagrangian TSVR (LTSVR) models have been used for the regression problems. ε-TSVR and LTSVR models seek a pair of ε-insensitive proximal planes for generation of end regressor. However, SVR and TSVR based models have several drawbacks- i) SVR model is computationally inefficient compared to the TSVR based models. ii) Twin SVM based models involve the computation of matrix inverse which is intractable in real world scenario's. iii) Both TSVR and LTSVR models are based on empirical risk minimization principle and hence may be prone to overfitting. iv) TSVR and LTSVR assume that the matrices appearing in their formulation are positive definite which may not be satisfied in real world scenario's. To overcome these issues, we formulate improved least squares twin support vector regression (ILSTSVR). The proposed ILSTSVR modifies the TSVR by replacing the inequality constraints with the equality constraints and minimizes the slack variables using squares of L2 norm instead of L1. Also, we introduce a different Lagrangian function to avoid the computation of matrix inverses. We evaluated the proposed ILSTSVR model on the subjects including cognitively healthy, mild cognitive impairment and Alzheimer's disease for brain-age estimation. Experimental evaluation and statistical tests demonstrate the efficiency of the proposed ILSTSVR model for brain-age prediction.

Price Prediction of Bitcoin Based on Adaptive Feature Selection and Model Optimization

Bitcoin is one of the most successful cryptocurrencies, and research on price predictions is receiving more attention. To predict Bitcoin price fluctuations better and more effectively, it is necessary to establish a more abundant index system and prediction model with a better prediction effect. In this study, a combined prediction model with twin support vector regression was used as the main model. Twenty-seven factors related to Bitcoin prices were collected. Some of the factors that have the greatest impact on Bitcoin prices were selected by using the XGBoost algorithm and random forest algorithm. The combined prediction model with support vector regression (SVR), least-squares support vector regression (LSSVR), and twin support vector regression (TWSVR) was used to predict the Bitcoin price. Since the model’s hyperparameters have a great impact on prediction accuracy and algorithm performance, we used the whale optimization algorithm (WOA) and particle swarm optimization algorithm (PSO) to optimize the hyperparameters of the model. The experimental results show that the combined model, XGBoost-WOA-TWSVR, has the best prediction effect, and the EVS score of this model is significantly better than that of the traditional statistical model. In addition, our study verifies that twin support vector regression has advantages in both prediction effect and computation speed.

Robust Twin Support Vector Regression with Smooth Truncated Hε Loss Function

Robust Twin Support Vector Regression with Smooth Truncated Hε Loss Function

End‐Point Dynamic Control Model for 260 Tons BOF Steelmaking Based on PWWTSVR and LWOA

The control of end‐point carbon content and temperature is very important for basic oxygen furnace (BOF) steelmaking. Herein, to improve the control precision of the BOF end‐point, a new dynamic control model of the BOF end‐point is proposed. First, based on the samples of low‐carbon steel collected from the steel plant, a prediction model of BOF end point is established by using projection wavelet weighted twin support vector regression (PWWTSVR). It is indicated in the simulation results that the error bound of carbon content and temperature are 0.005 wt% and 10 °C, respectively, and the hit rates of the prediction models are 92% and 90%. Moreover, a double hit rate of 84% is higher than the other five prediction models. Based on the PWWTSVR prediction model, a Lévy‐flying whale optimization algorithm (LWOA) is used to optimize the objective function to establish the control model. The control model shows that the mean absolute error of the end‐blow oxygen volume calculated by the model is 75.0201 Nm3, which is smaller by comparing the other two control models. The proposed control model can provide efficient guidance for on‐site smelting personnel.

Time-Delay Twin Support Vector Regression- Based Adaptive Predistorter for LED Nonlinearity in Visible Light Communications

The bit error rate (BER) of visible light communications (VLC) based on light emitting diodes (LEDs) has been limited due to the nonlinearity of LED. Artificial neural networks (ANNs) predistorters were applied to mitigate the impacts of LED nonlinearity. However, none of them can efficiently approach the performance of linear VLC systems due to the overfitting issue. In this paper, we for the first time propose a predistorter with adaptive indirect learning architecture based on amplitude time-delay twin support vector regression (ATD-TSVR) for LED nonlinearity in orthogonal frequency division multiplexing(OFDM)-based VLC systems. The experimental results demonstrate the signal distortion caused by LEDs as the major nonlinear source. Moreover, the simulation results show superior BER, amplitude distortion (AM/AM) curves, Power Spectral Density(PSD) and constellation plots in an 80Mbit/s OFDM nonlinear VLC system. Meanwhile, as compared to the traditional SVR approach, the CPU training time of ATD-TSVR can be reduced by more than four times. The adaptive pre-distortion method herein is generally applicable to broadband VLC systems and also proves the application prospect and effectiveness of TSVR in VLC system.

Robust and Sparse Dual Tree Complex Wavelet Transform-Based Twin Support Vector Regression for 5G InH and V2I Communications

Robust and Sparse Dual Tree Complex Wavelet Transform-Based Twin Support Vector Regression for 5G InH and V2I Communications

On wavelet transform based convolutional neural network and twin support vector regression for wind power ramp event prediction

Power produced from renewable energy sources carbon negative and promises an increased reliability for grid integration. Wind energy sector globally has an installed capacity of over 650 GW and promises to grow substantially. In this work, wind power ramp events that arise from sudden change in wind power are studied. Forecasting wind power ramp events is an important problem statement in the current wind power industry. Wind power integration to utility grid is impacted by the forecast accuracy. To improve the reliability and security, wind speed and power forecasts are extensively studied. Ramp events are predicted in this manuscript using hybrid techniques employing wavelet decomposition transform in tandem with convolutional neural network and twin support vector machines. Wind speed datasets from Spain, Germany and Argentina are considered and error metrics are computed. It is observed that CNN based method is 28.76% and 26.43% superior from wavelet based random forest and TSVR method.

Robust Least Squares Twin Support Vector Regression With Adaptive FOA and PSO for Short-Term Traffic Flow Prediction

Accurate short-term traffic flow prediction plays an important role in the field of modern Intelligent Transportation Systems. Since various uncontrollable factors ( <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">e.g.</i> weather, traffic jams or accidents), collected traffic data inevitably contain outliers. This makes it challenge to achieve satisfactory results for traffic flow prediction. Least Squares Twin Support Vector Regression (LSTSVR) has been shown to provide a powerful potential in nonlinear prediction problems. This is especially true when using appropriate heuristic algorithms to determine the parameters of nonlinear LSTSVR. In view of this, a novel LSTSVR model based on the robust <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\text{L}_{2,\mathrm {p}}$ </tex-math></inline-formula> -norm ( <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$0&lt; p\le 2$ </tex-math></inline-formula> ) distance is proposed to alleviate the negative effect of traffic data with outliers, called PLSTSVR. An iterative algorithm is designed to solve the optimization problem of PLSTSVR, which has great potential for solving other relevant optimization problems. To search the parameters of constructed PLSTSVR, this paper constructs two traffic flow prediction models based on PLSTSVR and heuristic algorithms (Fruit Fly Optimization Algorithm and Particle Swarm Optimization), called PLSTSVR-FOA and PLSTSVR-PSO. Extensive experiments demonstrate that the constructed models are more effective and robust than other competing models in various experimental settings.