Multi-output Support Vector Machine Research Articles

Uncertainty-Based Performance Prediction and Optimization of High-Fluidization Cement Grouting Material Using Machine Learning and Bayesian Inference

In pavement engineering, cement grouting material is widely used to pour into large void asphalt concrete to prepare semi-flexible composite mixtures. It plays an essential role in the performance of the semi-flexible composite mixture. To meet specific engineering requirements, various additives are mixed into the grouting material to improve the physical and mechanical properties. As a result, the uncertainty of the grouting material is also more significant as the complexity of material composition increases during the material design. It will bring some unknown risks for the engineering application. Hence, it is necessary to quantize the uncertainty during the material design of the grouting material and evaluate the reliability of the material formula. In this study, a novel framework of material design was developed by combing the Multioutput support vector machine (MSVM), Bayesian inference, and laboratory experiments. The MSVM was used to approximate and characterize the complex and nonlinear relationship between the grouting material formula and its properties based on laboratory experiments. The Bayesian inference was adopted to deal with the uncertainty of material design using the Markov Chain Monte Carlo. An optimized formula of the cement grouting material is obtained based on the developed framework. Experimental results show that the optimized formula improves engineering properties and performance stability, especially early strength. The developed framework provides a helpful, valuable, and promising tool for evaluating the reliability of the material design of the grouting material considering the uncertainty.

Reliability sensitivity analysis of thermal protection system

This paper carries on a reliability sensitivity analysis on the non-ablative thermal protection system (TPS) of spacecraft during the conceptual design. In the previous work on probabilistic estimation of TPS, the temperature dependency of material properties has not yet been investigated. In this paper, however, the temperature dependency of material properties is characterized and considered during the thermal analysis and reliability sensitivity analysis. Compared to general black-box problems, three special challenges of uncertainty analysis for TPS in real practice are a generally high dimension and multiple outputs on massive meshing nodes, a high level of reliability design target, and a fast evaluation process due to the requirement of the conceptual design. In order to cope with these challenges, a unified reliability sensitivity analysis methodology including multi-input and multi-output support vector machines (MIMO-SVMs), a space-partition (SP) method, and a generalized subset simulation (GSS) is proposed for the conceptual design of TPS with temperature-dependent materials. MIMO-SVMs are used to approximate the thermal responses to save calculation costs. The variance-based global sensitivity indices are calculated by SP to make full use of the information within samples. Based on the sensitivity indices, a dimension reduction process is introduced. In the reduced space, GSS is used to simultaneously evaluate all the failure probabilities by fully exploring the correlation among all the LSFs. Two application examples including a lifting body vehicle model and a spacecraft model are used to demonstrate the performance of the proposed methodology.

A reduced order model based on machine learning for numerical analysis: An application to geomechanics

Numerical methods are very important in geotechnical and geological engineering. This study presents a reduced-order numerical model to approximate the displacement and stress field in geotechnical and geological engineering contexts by combining numerical methods, proper orthogonal decomposition (POD), and multi-output support vector machine (MSVM). Snapshots were generated using Latin hypercube sampling. POD was used to compute POD-based vectors and their coefficients. Training samples were constructed from the numerical model and POD coefficients input. The MSVM algorithm was adopted to present the relationship based on these training samples. A reduced-order model was developed by predicting the POD coefficients using MSVM, and the displacement and stress field was then predicted based on the POD vectors and predicted POD coefficients. The proposed method was verified and demonstrated for a circular tunnel. The results show the displacement and stress fields are in excellent agreement with the analytical solution and with the numerical solution, and the predicted deformations are consistent with rock mechanics theory. The proposed method predicts the deformation and mechanical behavior of geomaterials well and may be used to replace numerical models for back analysis, for optimal design, and for uncertainty analysis in geotechnical and geological engineering, all of which involve repeated computation.

Multivariable Case Adaptation Method of Case-Based Reasoning Based on Multi-Case Clusters and Multi-Output Support Vector Machine for Equipment Maintenance Cost Prediction

Case adaptation is crucial for case-based reasoning (CBR) because the solutions of old cases are not always the ideal answer for the encountered new problem. It is employed to solve new problems by utilizing the adaptation knowledge extracted from similar ones encountered in the past. The traditional adaptation method solves a new problem in the principle of k- nearest neighbor ( $k$ -NN), and the adaptation model was built based on $k$ similar cases. Yet, the $k$ similar cases retrieved by new case may locate in different case clusters in the case base composed of multiple case clusters. This article presents a new case adaptation method by the combination of multi-adaptation engines from different case clusters to improve the adaptation accuracy. First, the input and output of the cluster-based adaptation engine are established from the old cases to distill the adaptation knowledge in each case cluster. Then, the multivariable CBR adaptation engine based on multiple-output support vector regression (MSVR) is built for case adaptation. Furthermore, inspired by the fact that the training sample which contains two closet cases can provide more useful information than others, and reduce the impact of outliers, this study adds the hybrid weight into MSVR, and allocates high weights to the information provided by such high sample density and similarity samples during multi-dimensional regression estimation. Finally, the solution of the target case is gathered by incorporating the output of different adaptation engines. The proposed method was applied to the equipment maintenance cost prediction and compared with traditional statistical-based and machine learning-based methods. Empirical comparison results indicated that the proposed adaptation method could achieve the best performance by utilizing the adaptation knowledge in different clusters under multi-case clusters environment.

Prediction of shear strength parameters of hydrocarbon contaminated sand based on machine learning methods

ABSTRACT The objective of this paper is to predict the effect of hydrocarbon contamination on the shear strength parameters of sand by using various machine learning platforms. Multilayer perceptron, support vector machine, random forest, gradient boosting method, and multi-output support vector machine were methods used to predict the hydrocarbon contamination impacts on the internal friction angle and cohesion of contaminated sand. Random forest exhibited the best results for cohesion, whereas, for the friction angle, the gradient boosting method outperformed other approaches. Moreover, the multi-output support vector machine yielded better results than those pertaining to a single support vector machine. Based on the sensitivity analyses, the most influencing parameter for cohesion prediction was the percentage of contamination for all the methods except for random forest where initial cohesion proved to be the primary governing factor, while the least influencing parameter was specific gravity for all methods. For the internal friction angle, the most influencing parameter was the initial friction angle for all methods except for gradient boosting where the percentage of contamination proved to be most controlling, whereas the least influencing factor was also specific gravity for all methods except for random forest where the initial cohesion was the least influencing parameter.

A novel spatio-temporal wind power forecasting framework based on multi-output support vector machine and optimization strategy

The integration of a large number of wind farms poses big challenges to the secure and economical operation of power systems, and ultra-short-term wind power forecasting is an effective solution. However, traditional approaches can only predict an individual wind farm power at a time and ignore the spatio-temporal correlation of wind farms. In this paper, a novel ultra-short-term forecasting framework based on spatio-temporal (ST) analysis, multi-output support vector machine (MSVM) and grey wolf optimizer (GWO) which defined ST-GWO-MSVM model is proposed to predict the output wind power from multiple wind farms; the ST-GWO-MSVM model includes data analysis stage, parameters optimization stage, and modeling stage. In the data analysis stage, the person correlation coefficient and partial autocorrelation function are used to analyze the spatio-temporal correlation of wind power. In the parameters optimization stage, to avoid obtaining the unreliable forecasting results due to the parameters are chosen empirically, the GWO algorithm is used to optimize the kernel function parameters of the MSVM model. In the modeling stage, an innovative forecasting model with optimal parameter of MSVM is proposed to predict the output wind power of 15 wind farms. Results show that the performance of ST-GWO-MSVM is better than other benchmark models in terms of multiple-error metrics including fractional bias, direction accuracy, and improvement percentages.

Wind weather prediction based on multi-output least squares support vector regression optimised by bat algorithm

As a kind of clean energy, wind energy is widely disseminated and has been widely researched. Compared with other methods, the support vector machine algorithm is more logical. Least squares support vector machine can improve training efficiency. Therefore, the method of multi-output least squares support vector regression is used to forecast the wind speed and wind direction in this paper. The bat algorithm is simple in structure and easy to understand. It has been applied to solve optimisation problems with MSVR. Compared with single output support vector machines, multi-output support vector machine readily solves problems of complex structure. The simulation model is established to predict the value of wind speed and wind direction by using different algorithms. The simulation results show that the multi-output least squares support vector machines prediction model based on bat optimisation algorithm has better feasibility and effectiveness.

Wind weather prediction based on multi-output least squares support vector regression optimised by bat algorithm

As a kind of clean energy, wind energy is widely disseminated and has been widely researched. Compared with other methods, the support vector machine algorithm is more logical. Least squares support vector machine can improve training efficiency. Therefore, the method of multi-output least squares support vector regression is used to forecast the wind speed and wind direction in this paper. The bat algorithm is simple in structure and easy to understand. It has been applied to solve optimisation problems with MSVR. Compared with single output support vector machines, multi-output support vector machine readily solves problems of complex structure. The simulation model is established to predict the value of wind speed and wind direction by using different algorithms. The simulation results show that the multi-output least squares support vector machines prediction model based on bat optimisation algorithm has better feasibility and effectiveness.

Location layout design of aircraft parts assembly based on MSVR

Location layout of aircraft assembly is an important factor affecting product quality. Most of the existing re-searches use the combination of finite element analysis and intelligent algorithm to optimize the location layout, which are limited by numerical simulation accuracy and the selection and improvement of intelligent algorithms. At present, the analysis and decision-making technology based on field data is gradually applied in aircraft manufacturing. Based on the perception data of intelligent assembly unit of aircraft parts, a regression model of multi-input and multi-output support vector machine with Gauss kernel function as radial basis function is established, and the hyperparameters of the model are optimized by hybrid particle swarm optimization genetic algorithm (PSO-GA). GA-MSVR, PSO-MSVR and PSOGA-MSVR model are constructed respectively, and their results show that PSOGA-MSVR model has the best performance. Finally, the design of the aircraft wing location layout is taken as an example to verify the effectiveness of the method.

Multi-output support vector machine for regional multi-step-ahead PM2.5 forecasting

Air quality deteriorates fast under urbanization in recent decades. Reliable and precise regional multi-step-ahead PM2.5 forecasts are crucial and beneficial for mitigating health risks. This work explores a novel framework (MM-SVM) that combines the Multi-output Support Vector Machine (M-SVM) and the Multi-Task Learning (MTL) algorithm for effectively increasing the accuracy of regional multi-step-ahead forecasts through tackling error accumulation and propagation that is commonly encountered in regional forecasting. The Single-output SVM (S-SVM) is implemented as a benchmark. Taipei City of Taiwan is our study area, where three types of air quality monitoring stations are selected to represent areas imposed with high traffic influences, high human activities and commercial trading influences, and less human interventions close to nature situation, respectively. We consider forecasts of PM2.5 concentrations as a function of meteorological and air quality factors based on long-term (2010–2016) observational datasets. Firstly, the Kendall tau coefficient is conducted to extract key spatiotemporal factors from regional meteorological and air quality inputs. Secondly, the M-SVM model is trained by the MTL to capture non-linear relationships and share correlation information across related tasks. Lastly, the MM-SVM model is validated using hourly time series of PM2.5 concentrations as well as meteorological and air quality datasets. Regarding the applicability of regional multi-step-ahead forecasts, the results demonstrate that the MM-SVM model is much more promising than the S-SVM model because only one forecast model (MM-SVM) is required, instead of constructing a site-specific S-SVM model for each station. Moreover, the forecasts of the MM-SVM are found better consistent with observations than those of any single S-SVM in both training and testing stages. Consequently, the results clearly demonstrate that the MM-SVM model could be recommended as a novel integrative technique for improving the spatiotemporal stability and accuracy of regional multi-step-ahead PM2.5 forecasts.

A modularized case adaptation method of case-based reasoning in parametric machinery design

Case adaptation is fundamentally to successfully applying case-based reasoning (CBR) in parametric machinery design, and support vector machine (SVM)-based adaptation is a promising method for CBR adaptation. But the standard formulation of SVM can only be used as a univariate modeling technique due to its inherent single-output structure, which result in the construction of different SVM-based adaptation engine for each solution element adaptation, and such engines could ignore the effects of the mutual parameter relationships for the adaptation results. This paper focuses on the multivariable adaptation problem in CBR adaptation, and proposes a modularized adaptation method by integrating with multiply relational analysis, case parameter clustering and adaptation engine construction. Firstly, the hidden parameter relationships between problem and solution (P–S), problem and problem (P–P), and solution and solution (S–S) parameters are extracted from old cases, then these parameters are clustered into several parameter clustering (PC) modules in terms of their internal relationships. Finally, multi-output SVM (MSVM) is used to build the adaptation engine for each PC module. This method not only improves the performance of SVM-based adaptation by utilizing the mutual parameter relationships, but also reduces the computational expense of MSVM-based adaptation by partitioning the only one adaptation engine into several sub-engines. Actual design examples are introduced to illustrate the process of modularized adaptation, and the empirical experiments in the different examples are carried out to validate the superiority of our proposed method. Through comparing the adaptation accuracies with those provided by other classical neuro-adaptation methods, the modularized adaptation is proved to be a feasible method for case adaptation.

Back Analysis of Rock Hydraulic Fracturing by Coupling Numerical Model and Computational Intelligent Technology

Hydraulic fracturing is widely used to determine in situ stress of rock engineering. In this paper we propose a new method for simultaneously determining the in situ stress and elastic parameters of rock. The method utilizing the hydraulic fracturing numerical model and a computational intelligent method is proposed and verified. The hydraulic fracturing numerical model provides the samples which include borehole pressure, in situ stress, and elastic parameters. A computational intelligent method is applied in back analysis. A multioutput support vector machine is used to map the complex, nonlinear relationship between the in situ stress, elastic parameters, and borehole pressure. The artificial bee colony algorithm is applied in back analysis to find the optimal in situ stress and elastic parameters. The in situ stress is determined using the proposed method and the results are compared with those of the classic breakdown formula. The proposed method provides a good estimate of the relationship between the in situ stress and borehole pressure and predicts the maximum horizontal in situ stress with high precision while considering the influence of pore pressure without the need to estimate Biot’s coefficient and other parameters.

Structural reliability analysis of multiple limit state functions using multi-input multi-output support vector machine

Selecting and using an appropriate structural reliability method is critical for the success of structural reliability analysis and reliability-based design optimization. However, most of existing structural reliability methods are developed and designed for a single limit state function and few methods can be used to simultaneously handle multiple limit state functions in a structural system when the failure probability of each limit state function is of interest, for example, in a reliability-based design optimization loop. This article presents a new method for structural reliability analysis with multiple limit state functions using support vector machine technique. A sole support vector machine surrogate model for all limit state functions is constructed by a multi-input multi-output support vector machine algorithm. Furthermore, this multi-input multi-output support vector machine surrogate model for all limit state functions is only trained from one data set with one calculation process, instead of constructing a series of standard support vector machine models which has one output only. Combining the multi-input multi-output support vector machine surrogate model with direct Monte Carlo simulation, the failure probability of the structural system as well as the failure probability of each limit state function corresponding to a failure mode in the structural system can be estimated. Two examples are used to demonstrate the accuracy and efficiency of the presented method.

Inverse analysis of geomechanical parameters by the artificial bee colony algorithm and multi-output support vector machine

Geomechanical parameters play a very important role in geotechnical engineering design and construction. Inverse analysis provides a powerful tool to characterize geomechanical parameters based on the behaviour of rock under certain boundary conditions. In this paper, a new method for inverse analysis that takes advantage of multi-output support vector machine (MSVM) and artificial bee colony (ABC) is proposed and applied to solving a real field problem. MSVM is used to map the relationship between geomechanical parameters and displacements. ABC is adopted in inverse analysis to find the optimal geomechanical parameters. The proposed method is used to recognize the parameters of the permanent shiplock slope of the Three Gorges in China. Results show that the proposed method can effectively determine geomechanical parameters.

Identifying geomechanical parameters of high cut rock slopes by an improved multi-output support vector machine method

Mechanical parameters of geomaterials are very important to slope stability assessment, design, and safety construction. Back analysis is a practical engineering tool which is commonly used to identify geomechanical parameters based on monitored displacement during slope excavation. The key problem is to establish the nonlinear relationship between geomechanical parameters and the monitored displacements at different excavation steps. This paper presents an improved multi-output support vector machine (MSVM) method which integrates the artificial bee colony algorithm (ABC), numerical analysis, and updated displacement monitoring. MSVM was used to map the complex relationship between geomechanical parameters and slope displacement increments. Training samples were chosen following a Latin hypercube model and their data were applied to numerical model. ABC formed the back analysis, searching for optimal geomechanical parameters based on the MSVM model and field-monitored displacement. The proposed method was applied to a high abutment steep rock slope of the Longtan hydropower station in China and successfully represented the relationship between geomechanical parameters and displacement increments. Updating the displacements at different excavation steps improved the precision of recognition. Reasonable mechanical parameters were identified and adopted in the slope stability analysis.

Probabilistic back analysis based on Bayesian and multi-output support vector machine for a high cut rock slope

Uncertainty of geomechanical parameters is an important consideration for rock engineering and has a very important influence on safety evaluation, design, and construction. Back analysis is a common method of determining geomechanical parameters but traditional deterministic back analysis cannot allow for consideration of this uncertainty. In this study, a new probabilistic back analysis method is proposed that integrates Bayesian methods and a multi-output support vector machine (B–MSVM). In this B–MSVM back analysis method, Bayesian was used to deal with the uncertainty of geomechanical parameters and a multi-output support vector machine (MSVM) was adopted to build the relationships between displacements and those parameters. The proposed method was applied to a high abutment rock slope at the Longtan hydropower station, China. At Longtan, the uncertainty of the two types of geomechanical parameters, Young's modulus and lateral pressure coefficients of in situ stress, were modeled as random variables. Based on the parameters identified by probabilistic back analysis, the computed displacements agreed closely with the measured displacement data monitored in the field. The result showed that B–MSVM presented the uncertainty of the geomechanical parameters reasonably. Further study indicated that the performance of B–MSVM could be improved greatly by updating field monitoring information regularly. The proposed method provides a significant new approach for probabilistic back analysis and contributes to the determination of realistic geomechanical parameters.

The combination of adaptive database SDM and multi-output SVM for eddy current testing

Purpose – Eddy current testing (ECT) is a nondestructive testing method for the detection of flaws that uses electromagnetic induction to find defects in conductive materials. In this method, eddy currents are generated in a conductive material by a changing magnetic field. A defect is detected when there is a disruption in the flow of the eddy current. The purpose of this paper is to develop a new noniterative inversion methodology for detecting degradation (defect characterization) such as cracking, corrosion and erosion from the measurement of the impedance variations. Design/methodology/approach – The methodology is based on multi-output support vector machines (SVM) combined with the adaptive database schema design method (SDM). The forward problem was solved numerically using finite element method (FEM), with its accuracy experimentally verified. The multi-output SVM is a statistical learning method that has good generalization capability and learning performance. FEM is used to create the adaptive database required to train the multi-output SVM and the genetic algorithm is used to tune the parameters of multi-output SVM model. Findings – The results show the applicability of multi-output SVM to solve eddy current inverse problems instead of using traditional iterative inversion methods which can be very time-consuming. With the experimental results the authors demonstrate the accuracy which can be provided by the multi-output SVM technique. Practical implications – The work allows extending the capability of the experimentation ECT defect characterization system developed at LGEP. Originality/value – A new inversion method is developed and applied to ECT defect characterization. This new concept introduces multi-output SVM in the context of ECT. The real data together with estimated one obtained by multi-output SVM model are compared in order to evaluate the effectiveness of the developed technique.

A Multi-Dimensional Method for Nodal Load Forecasting

The intelligent dispatch and control of future smart grid demands grasping of any nodal load pattern in the general great grid, therefore to realize the load forecasting of any nodal load is quite important. To solve this problem, focusing on overcoming the weakness of isolated nodal load forecasting and based on the correlation analysis, this paper proposes a multi-dimensional nodal load forecast system and corresponding method for effective prediction of any nodal load of the grid. This system includes topology partitioning of the grid energy flow according to layers and regions, basic forecasting unit composed of each layer’s total amount of load and its nodal loads, and combination forecasting for any node. The forecasting method is based on techniques including the multi-output least square support vector machine, Kalman filtering and the approximate optimal prediction. A case study shows that the multi-dimensional nodal load forecasting model helps to improve the forecasting accuracy, and has practical prospects.

Research of the Power Control for a MCFC/GT Hybrid System

Molten carbonate fuel cell/ gas turbine hybrid power system (MCFC/GT) output power should real-time response the demand of load. The auto-tuning ZieglerNichols tuned PI controller (AZNPIC) as a feedback controller for power corrective action is added to the hybrid power system. The optimal setpoints and feed forward control inputs of the controller are given by Multi-output Support Vector Machine Regression (MSVR) predictor for the hybrid system. Simulation results show that the power of hybrid system can be effectively close to the desired setpoints based on the control method.

A fast and robust model selection algorithm for multi-input multi-output support vector machine

Multi-Input Multi-Output (MIMO) regression estimation problems widely exist in engineering fields. As an efficient approach for MIMO modeling, multi-dimensional support vector regression, named M-SVR, is generally capable of obtaining better predictions than many traditional methods. However, M-SVR is sensitive to the perturbation of hyper-parameters when facing small-scale sample problems, and most of currently used model selection methods for conventional SVR cannot be applied to M-SVR directly due to its special structure. In this paper, a fast and robust model selection algorithm for M-SVR is proposed. Firstly, a new training algorithm for M-SVR is proposed to reduce efficiently the numerical errors in training procedure. Based on this algorithm, a new leave-one-out (LOO) error estimate for M-SVR is derived through a virtual LOO cross-validation procedure. This LOO error estimate can be straightway calculated once a training process ended with less computational complexity than traditional LOO method. Furthermore, a robust implementation of this LOO estimate via Cholesky factorization is also proposed. Finally, the gradients of the LOO estimate are calculated, and the hyper-parameters with lowest LOO error can be found by means of gradient decent method. Experiments on toy data and real-life dynamical load identification problems are both conducted, demonstrating comparable results of the proposed algorithm in terms of generalization performance, numerical stability and computational cost.