Multi-output Support Vector Regression Research Articles

A critical review of multi-output support vector regression

Single-output regression is a widely used statistical modeling method to predict an output based on one or more features of a datapoint. If a dataset has multiple outputs, they can be predicted independently from each other although this disregards potential correlations and thus may negatively affect the predictive performance. Therefore, multi-output regression methods predict multiple outputs simultaneously. One way to approach single-output regression is by using methods based on support vectors such as support vector regression (SVR) or least-squares SVR (LS-SVR). Based on these two, previous works have devised multi-output support vector regression methods. In this review, we introduce a unified notation to summarize the single-output support vector regression methods SVR and LS-SVR as well as state-of-the-art multi-output support vector regression methods. Furthermore, we implemented a workflow for subject- and record-wise bootstrapping and nested cross-validation experiments, which we used for an exhaustive evaluation of all single- and multi-output support vector regression methods on synthetic and non-synthetic datasets. Although the reviewed papers claim that their multi-output methods improve regression performance, we find that none of them outperform both single-output methods SVR and LS-SVR for various reasons. Due to these results, we reflected about the general concept of support vector regression and then concluded that support vector regression methods do not appear to be suitable for the task of multi-output regression.

Quantitative characterization of microstructures and elastic properties in plasma sprayed coatings based on “Ultrasonic – Elasticity – Microstructure” interaction mechanism

In plasma sprayed coatings, pores are the primary microstructure, and their features such as porosity, shape, and orientation significantly influence the thermal insulation and elastic properties of the coatings. Therefore, the ultrasonic characterization of pore feature parameters plays a crucial role in guiding the processing and predicting the lifespan of coatings. However, the coupling of multiple pore feature parameters leads to ill-posed problem and nonlinearity between these parameters and the ultrasonic feature signals, making it challenging to identify multiple pore feature parameters accurately. In this study, based on the “ultrasonic - elasticity - structure” interaction mechanism, a novel strategy for the ultrasonic quantitative identification of pore feature parameters and elastic constants in plasma sprayed coatings is proposed.Firstly, a random pore model (RPM) was developed to reveal the complex morphology of pores in plasma sprayed coatings. The porosity, average aspect ratio, orientation factor, and contour roughness factor were statistically analyzed for 15 sets of constructed RPMs, followed by ultrasonic finite element simulations. Then, a sensitivity matrix (SM) inversion method is developed for the “ultrasonic - elasticity” relationship by accurately measuring the elastic constants of the coatings through ultrasonic wave velocity inversion at multiple angles, yielding a relative error of only 0.57%. Lastly, a Multi-output Support Vector Regression machine learning model optimized by genetic algorithm (GA-MSVR) was proposed to establish the “elasticity - microstructure” relationship and identify multiple pore feature parameters based on elastic constants. The maximum relative errors for the four pore feature parameters are found to be 12.28%, 6.35%, 18.87%, and 7.62%, respectively, demonstrating the feasibility of the proposed strategy. Furthermore, the sensitivity matrix was employed to explain the cause for the larger relative error of the orientation factor.In summary, a challenging task of quantitatively identifying pore feature parameters in plasma sprayed coatings is addressed by proposing a novel solving strategy based on the “ultrasonic - elasticity - microstructure” interaction mechanism. The developed RPM, SM inversion method, and GA-MSVR model contribute to the accurate and comprehensive identification of pore feature parameters. The results obtained validate the feasibility of the proposed strategy.

Multi-Output Regression Indoor Localization Algorithm Based on Hybrid Grey Wolf Particle Swarm Optimization

In the evolving landscape of device-free localization techniques, Wi-Fi channel state information (CSI) emerges as a pivotal tool for environmental sensing. This study introduces a novel fingerprint localization algorithm. It employs an improved Hybrid Grey Wolf Particle Swarm Optimization (IPSOGWO) in combination with Multi-Output Support Vector Regression (MSVR) to enhance indoor positioning accuracy. To counteract the limitations of standard DBSCAN and PCA in noise reduction and feature extraction from complex nonlinear data, we propose an adaptive denoising algorithm based on spatial clustering (A-DBSCAN) and an autoencoder to efficiently denoise and extract features from CSI amplitude to improve the localization accuracy. Additionally, we introduce a new position update strategy, bolstering the optimization efficiency of the PSOGWO algorithm. This refined approach is instrumental in determining the globally optimal hyperparameters in MSVR, leading to enhanced model prediction accuracy. Two indoor scenario experiments were conducted to evaluate our method, yielding average localization errors of 0.59 m and 1.12 m, marking an improvement in localization performance compared to existing methods.

Probability-based label enhancement for multi-dimensional classification

Multi-dimensional classification (MDC) assumes that each instance has multiple heterogeneous class spaces simultaneously, and each class variable describes the semantic information of instances from a specific dimension. Recent studies have proven that encoding heterogeneous class spaces into a special logical-label space and employing the label enhancement technique to learn latent real-number labels (i.e., label distributions) of instances is an effective strategy for MDC. However, the adopted label enhancement methods can result that data whose features are quite different to each other have similar label distributions. To tackle this problem, we propose a novel probability-based label enhancement approach for MDC. Specifically, manifold structures of the feature and label distribution spaces are transformed into two different probability distributions, and we expect them to be close. Subsequently, it makes label distributions of samples whose features have large differences be more differentiated. Moreover, the logical-label mapping and reconstruction terms are designed to preserve the intrinsic information from the logical-label space. Besides, an improved multi-output support vector regression is developed as the prediction model, where we introduce mean squared error to reduce the risk of model underfitting. Experimental results on ten benchmark datasets clearly validate the superiority of our method over state-of-the-art MDC baselines.

Back analysis of rock mass parameters in tunnel engineering using machine learning techniques

Efficient determination of the rock mass properties is vitally important for calculating and evaluating tunnel stability in tunnel engineering. The back analysis method has been widely used as an indirect method for determining rock mass parameters based on field measurements. However, most back analysis methods are generally time-consuming for numerical simulation and are merely based on the measured displacement, which leads to the identification of rock mass parameters that cannot fully reflect the characteristics of the surrounding rock. To improve the accuracy of the estimation of rock mass parameters, this paper presents a back analysis method based on multi-output support vector regression (MSVR) and differential evolution (DE) algorithms. Firstly, the global sensitivity analysis of rock mass parameters is analyzed using the elementary effects method. Numerical simulation is then carried out to prepare training samples. DE algorithm is used to determine the optimum hyperparameters of MSVR. Based on the monitoring data, the rock mass properties of the selected sensitive parameters are estimated by the constructed MSVR model. A high-speed railway tunnel is utilized to demonstrate the effectiveness of the MSVR with the DE algorithm (DE-MSVR). The results show that the DE-MSVR with mixed monitoring data of vault settlement, convergence, and rock mass stress has higher forecasting performance than these models with a single type of monitoring data. It is feasible to use the monitoring data at the early stages combined with the numerical simulation for parameter back analysis. Moreover, the comparison results show that the presented method exhibits higher prediction accuracy than the existing back analysis models.

Towards an Intelligent Model for Evaluating Serious Games

Serious games are effective educational tools used in higher education to provide practical learning opportunities to students. However, few research works have focused on evaluating serious games as a project for developing a tool dedicated to use in a formative context. This document proposes an intelligent evaluation model that not only allows for the evaluation of serious games but also facilitates their integration into teaching practice. The model is designed around four dimensions, and their measurement criteria are well defined. Fuzzy decision-making methods were used to weight the criteria, and supervised machine-learning algorithms were considered to minimize the evaluator’s bias. The proposed model provides a more objective and consistent solution for evaluating serious games, reducing the impact of evaluators’ biases and subjective preferences on the weightings of the different evaluation dimensions. The multi-output support vector regression (M-SVR) model can be used flexibly and adapted to different contexts and applications, offering a more effective and reliable solution for evaluating serious games.

Prediction of strip section shape for hot-rolled based on mechanism fusion data model

With the rise and vigorous development of artificial intelligence (AI) and data mining methodologies, machine learning (ML) has been successfully and widely used in industrial fields. For hot-rolled strips, both the rolling mechanism analyzing and multi-indicators integrated optimization are difficulties in hot rolling process. To overcome the limitations of multi-variable, nonlinear, and strong coupling in the process of hot rolling, and to improve the prediction accuracy of multi-output regression model, a novel model of rolling mechanism guided ML was developed. Mechanism data were calculated as added input features to participate in training process by way of dimensionality processing (DP). Simultaneously, genetic algorithm (GA) was adopted to realize multi-objective optimization of multi-output support vector regression (M-SVR) to further enhance the performance of model ultimately. Especially, the experimental results of root mean square error (RMSE) and correlation coefficient (R) of strip crown and thickness are 2.5218, 0.9891 and 2.1302, 0.9902, respectively, which fully demonstrates that this method does not limit the prediction accuracy but greatly improved the generalization ability.

System nonlinearity correction based on a multi-output support vector regression machine

In a fringe projection profilometry system, the phase error introduced by the projector's gamma distortion is the main source of errors. To overcome this problem, we present a phase compensation scheme for multi-dimensional harmonic coefficient prediction based on a multi-output support vector regression machine(M-SVR), The scheme first constructs a significant characteristic relationship between phase probability density function (PDF) and phase multi-harmonic coefficients, creates simulation data without a priori knowledge, constructs a data set with a certain sample size, and then trains the M-SVR model. The trained M-SVR model is used to capture the potential features of the experimental distorted phase and output the multi-dimensional harmonic parameters with nonlinear relationships, followed by error compensation of the distorted phase using an immobile point iteration algorithm for the purpose of correcting the system nonlinearity. We demonstrate the validity and stability of the model through simulation and experimental trials. Most importantly, the preprocessed M-SVR model also has the potential to participate in error correction of other measurement experiments with reasonable sample and hyperparameter settings, which greatly saves the time and cost of multiple experiments.

Cluster-based industrial KPIs forecasting considering the periodicity and holiday effect using LSTM network and MSVR

Key performance indicators (KPIs) are key indicators to measure the status and performance of various facilities, time, and resources in the process of production and life. Forecasting industrial KPIs helps in analyzing the future state to make better production and scheduling decisions. However, due to the complexity of industrial production activities, forecasting industrial KPIs becomes an important and challenging task. In this paper, considering the periodicity and holiday effect of data, a cluster-based industrial KPIs forecasting method using long short-term memory (LSTM) network and multi-output support vector regression (MSVR) is proposed. First, based on the knowledge of production and scheduling, the original daily data is divided into workdays, off-days, and holidays, and K-shape clustering divides holidays into multiple sub-series according to the characteristics of distinct holidays. Then, for the new prediction task, the most suitable historical sub-series is selected through a matching strategy combined with date information. Finally, a compatible model is adaptively selected for forecasting according to the type of sub-series. Specifically, LSTM network is employed to forecast the KPIs on workdays and off-days with a large amount of data, and MSVR is adopted to forecast the KPIs on holidays with few samples. To assess the effectiveness of the proposed method, experiments are conducted in two public datasets with several state-of-the-art methods. The results reveal the superiority of the proposed method in industrial KPIs forecasting.

Improving the forecasting accuracy of interval-valued carbon price from a novel multi-scale framework with outliers detection: An improved interval-valued time series analysis mode

Accurate carbon price forecasting can provide policymakers with a reasonable basis for carbon pricing. Interval-valued carbon price forecasting could provide sufficient information compared with real-valued carbon price time series prediction. On the other hand, current interval-valued carbon price forecasting has major challenges including data complexity, outliers, and the selection of forecasting methods, which make the forecasting results with great uncertainty and instability. To address these issues, this paper proposes an interval-valued carbon price forecasting method based on new data processing techniques, and discusses the effects of different combinations of interval variables on the forecasting results. We first established interval complete ensemble empirical mode decomposition with adaptive noise (ICEEMDAN) and interval outlier detection method (I-ksigma) to reduce the data complexity and identify outliers. Then, various neural network models include interval multilayer perceptron (IMLP), multi-output support vector regression (MSVR), long short-term memory network (LSTM), gated recurrent unit neural network (GRU), and convolution neural network (CNN) are chosen to conduct combination forecasting on the interval sub-modes produced by ICEEMDAN. The final results are obtained by summing the interval sub-modes. Finally, taking the carbon trading price in Hubei as the research object, the results show that the developed forecasting framework is superior to all comparison models in forecasting precision and stability. Furthermore, different combinations of interval variables (CRM, Minmax, L + 2R, and U-2R) lead to different decomposition results and outlier detection results, which finally affect the prediction results.

Intelligent Deployment Solution for Tabling Adapting Deep Learning

The deep learning object detection model can well extract the coordinate information of the separating point of the concentrate ore belt boundary line. However, with the drawbacks of acquired zonal image features, it is difficult to obtain the perfect prediction of the concentrate grade and recovery rate. In this study, by adapting deep learning semantic segmentation technique with DeepLab V3+, which can effectively extract multi-dimensional image features. The mapping relationship between image features and attributes of the Tabling equipment is achieved by constructing a multi-output support vector regression model optimized of a sparrow search algorithm (SSA-MSVR). The beneficiation indicators and operating parameters of the Tabling can be continually detected and optimized by an intelligent system that mainly includes image recognition softwares, automatic control units, data processing and communication workstations, and matching intelligent equipment, which accomplished the intelligent deployment solution of the Tabling beneficiation process.

Multi-output soft sensor modeling approach for penicillin fermentation process based on features of big data

The product quality indicators of the penicillin fermentation process have multiple semantics and are interrelated. There is a complex nonlinear mapping relationship between input characteristics and multiple-output objectives, and the time dependence is strong. As a result, the prediction accuracy of existing soft sensor models is poor, and it is difficult to meet the needs of industrial sites. To solve the above problems, this paper proposes a multi-output soft sensor modeling method for the penicillin fermentation process based on big data feature analysis. In this method, the process data is divided into several batches in order, and then the data features of multivariable and time-dependent datasets are extracted according to the deep sparse self-coding neural network method to realize the effective mining of the relationship between multivariable time series factors, based on the multi-output support vector regression method, several soft sensor models for different prediction targets are established. Meanwhile, to improve the Predictive performance of the soft sensor model, the improved black hole algorithm is used to optimize the model parameters. Finally, a simulation experiment is carried out based on the simulation dataset of the penicillin fermentation process to verify the effectiveness of the proposed method.

Strip Thickness and Profile–Flatness Prediction in Tandem Hot Rolling Process Using Mechanism Model‐Guided Machine Learning

This study is focused on the multivariable, nonlinear, strong‐coupling, and time‐varying characteristics of the quality control process of hot strip rolling. Using the rolling process data from a continuously variable crown (CVC) mill of a 1580 mm hot tandem rolling line, a new prediction model of the strip thickness, profile, and flatness in hot rolling based on rolling mechanism model‐guided machine learning (ML) is developed. The weighted processing (WP) method is used to fuse the rolling mechanism and process data, to improve the relationship between the strong correlation features and the model. Combined with nondominated sorting genetic algorithm III (NSGA‐III), multiple parameters of multioutput support vector regression (M‐SVR) are optimized. The results show that the root mean square error (RMSE), mean square correlation coefficient (R 2), and mean absolute error (MAE) of the strip thickness, crown, and flatness are 2.0159, 1.3191, and 0.9355; 0.9854, 0.9895, and 0.9873; and 16.601, 1.126, and 0.604, respectively. Moreover, the established method of data fusion rolling mechanism shows strong capability to improve the model prediction accuracy, increasing it by 60%. Thus, it can offer theoretical guidance for realizing accurate control of the quality of strips and improving the quality of hot‐rolled products.

A Fault-Tolerant Scheme Based on MSVR for Integrated Navigation

To enhance the reliability of integrated navigation systems, an intelligent fault-tolerant scheme based on multi-output support vector regression (MSVR) is proposed. Specifically, a novel fault detection function that is made up of the filtering innovations and the estimated ones is designed so that it can timely detect the faults that may occur. Meanwhile, MSVR is utilized to build the estimation model for more reliable estimations. The special construction of the fault detection function makes it very sensitive to faults. Additionally, a variable that symbolizes the confidence of measurement is defined and applied to realize fault isolation. The method effectively improves the navigation accuracy at the initial stage of the occurrence of a gradual fault. The results of experiments reveal that the proposed algorithm has a stronger fault-tolerant ability compared with the traditional algorithm. Concretely, when dealing with the artificial faults, the position precision in north and east directions are increased up to 70.06% and 67.78%, respectively, by applying the proposed fault-tolerant scheme, in comparison with the residual chi-square test.

Applying an MSVR Method to Forecast a Three-Degree-of-Freedom Soft Actuator for a Nonlinear Position Control System: Simulation and Experiments

In this article, a method used for tip-position coordinate control of a three-degree-of-freedom (DOF) soft actuator is proposed.In general, the behavior of pneumatic soft actuators is simple. However, the actuator, which consists of three artificial muscles, is capable of more complex motions compared to conventional soft actuators. By designing a model and control system that can handle multiple input patterns, various motions are possible. In addition, a machine learning technique called <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">multioutput support vector regression</i> ( <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">M-SVR</i> ) is used as a method to compensate for the complexity of multiple-input, multiple-output systems. First, a model that can be used to design a control system is offered. Then, a control system is designed, using the recommended model and machine learning approaches. Furthermore, the effectiveness of the proposed system is verified by experiments.

Nonparametric regression for interval-valued data based on local linear smoothing approach

In this paper, we propose an interval local linear method (ILLM) to fit the regression model with interval-valued explanatory and response variables. The proposed method has no restriction on the form of the regression function. Moreover, it reduces the boundary effect of interval kernel method. Some experimental studies including two simulations and four real datasets are examined to evaluate the proposed method. Experimental results show that our method has higher predictive accuracy than some existed methods, including the center and range method, the interval least absolute method, the interval kernel method, multi-output support vector regression and the interval multilayer perceptron.

Studying the Impact of Health Education on Student Knowledge and Behavior through Big Data and Cloud Computing

Artificial intelligence and big data, as emerging technologies that have attracted much attention in recent years, have broad application and development space in improving the development of intelligent and refined education in colleges and universities. The application of artificial intelligence and big data to the mental health education practice of college students has a very positive effect on accurately discovering and scientifically solving the mental health problems of college students. In order to combine big data and cloud computing platform organically, this paper introduces an intelligent algorithm based on multi-output support vector regression (MSVR) model and immune clone selection algorithm (ICSA). At the same time, we couple the two to obtain a new intelligent algorithm, namely, immune multiple output support vector regression (ICSA-MSVR) algorithm. Based on the prediction results of health education on students' knowledge and behavior by cloud computing platform, the necessary conditions for three intelligent algorithms to complete the task are summarized. Numerical experimental results show that ICSA-MSVR plays a role in both local search and global search, and is more effective in large-scale cloud computing task scheduling. In addition, in task scheduling, when the task completion time is short, ICSA-MSVR has a lower load imbalance than ICSA and MSVR, which can achieve better load balancing, and the load between virtual machines is closer. Finally, combined with the problems and the needs of students’ health education, suggestions are put forward to deepen the application of technology in students’ mental health education. This approach can provide corresponding ideas and reference methods for improving the scientificity, pertinence, and effectiveness of mental health education.

Prediction of Geometric Characteristics of Melt Track Based on Direct Laser Deposition Using M-SVR Algorithm.

Geometric characteristics provide an important means for characterization of the quality of direct laser deposition. Therefore, improving the accuracy of a prediction model is helpful for improving deposition efficiency and quality. The three main input variables are laser power, scanning speed, and powder-feeding rate, while the width and height of the melt track are used as outputs. By applying a multi-output support vector regression (M-SVR) model based on a radial basis function (RBF), a non-linear model for predicting the geometric features of the melt track is developed. An orthogonal experimental design is used to conduct the experiments, the results of which are chosen randomly as training and testing data sets. On the one hand, compared with single-output support vector regression (S-SVR) modeling, this method reduces the root mean square error of height prediction by 22%, with faster training speed and higher prediction accuracy. On the other hand, compared with a backpropagation (BP) neural network, the average absolute error in width is reduced by 5.5%, with smaller average absolute error and better generalization performance. Therefore, the established model can provide a reference to select direct laser deposition parameters precisely and can improve the deposition efficiency and quality.

A Novel Multi-Objective Process Parameter Interval Optimization Method for Steel Production

Customized small batch orders and sustainable development requirements pose challenges for product quality control and manufacturing process optimization for steel production. Building a multi-quality objective process parameter optimization method that converts the original single target optimization into multi-objective interval capability optimization has become a new method to ensure product quality qualification rate and reduce production costs. Aiming at the multi-quality objective control problem of plate products, we proposed a novel multi-objective process parameter interval optimization model (MPPIO) with equipment process control capability and parameter sensitive analysis. The multi-output support vector regression method was used to establish a multi-quality objective prediction model, which was settled as a verification model for the process parameter optimization results based on the particle swarm optimization algorithm (PSO). The process control capability functions of key parameters were fitted based on the real data in production. With these functions, each optimized particle of the classical PSO was converted into the particle beam of the MIPPO. The iteration process was weight controlled by calculating the Morris sensitivity between each input parameter and output index in the multi-quality objective prediction model, and finally the processing control window of each key parameter was determined according to the process parameter optimization results. The experimental results show that the MPPIO model can obtain the optimal parameter optimization results with the maximum processing capacity and meet the customized processing range requirements. The MPPIO model can reduce the difficulty of control and save production costs while ensuring the product properties is qualified.

A rule-based granular model development for interval-valued time series

As one type of time series, interval-valued time series (ITS) is encountered frequently in many fields such as finance, environment, agriculture and so on since it can describe the uncertainty and variability of observed variables. The modeling of ITS is an ongoing issue pursued by researchers. In this paper, a novel rule-based granular ITS model is proposed by considering the interval-valued data as interval information granules in the framework of granular computing (GrC). The development of the proposed granular model consists of three components with progressive relationship, that is, the generation of granular prototypes, the formation of initial granular model and the refinement of the initial granular model. Further, the detail computation of reasoning of the corresponding granular model is also given. The resulting granular models have not only interpretability but also an ability to process ITS containing linguistic variables. Numerical experiments on six financial datasets and two meteorological datasets from real world reveal the impact of the parameter involved in the proposed method on the resulting granular model, and also exhibit that the resulting granular model is superior to seven existing competitive models including interval Holt one, multi-output support vector regression one and interval arithmetic-based multilayer perceptron one etc.