Multi-output Least-squares Support Vector Regression Research Articles

Enhanced multi-task learning models for pile drivability prediction: Leveraging metaheuristic algorithms and statistical evaluation

As a crucial component in construction, piles find extensive use in transportation infrastructure. Drivability, a term commonly employed to gauge the ease of pile installation, encompasses various factors. To evaluate the drivability, various indexes, such as MCS (maximum compressive stresses), MTS (maximum tensile stresses) and BPF (blow per foot) are proposed. Despite the need for multiple indexes to evaluate pile drivability, these metrics often share underlying commonalities, as they collectively describe features of the hammer-pile-soil system comprehensively. Thus, leveraging multi-task learning techniques becomes advantageous for tackling the drivability prediction problem. This paper proposes two enhanced multi-task learning models improved from MLS-SVR (Multi-output least-squares support vector regression machines) and hybridized with metaheuristic algorithms. Based on 4072 pile installing samples, the models undergo development and hyperparameter optimization employing SA (Simulated Annealing) and YYPO (Yin-Yang-pair Optimization). Six statistical indexes are employed evaluate the accuracy of the models. The test results reveal the superiority of YYPO-MLS-SVR over SA-MLS-SVR. Furthermore, comparative analysis against single-task models from previous studies demonstrates the robust performance of hybridized MLS-SVR models, even when addressing multiple problems concurrently. This paper presents a novel approach for the multi-index evaluation of pile drivability.

Deformation health diagnosis of RCC dams considering construction interfaces based on monitoring data and numerical simulation

The health diagnosis procedure applied to determine the deformation of a roller compacted concrete (RCC) dam is different from that of a conventional concrete dam. Hence, in this study, a deformation health diagnosis model was established for an RCC dam considering the construction interfaces by combining the hydrostatic component simulated using ABAQUS (2016) with the temperature and aging components calculated using a statistical method. The combined method can help monitor the structural health of RCC dam and determine the physical meaning and statistical law of the deformation of RCC dams. During the simulation process, the geometric characteristics of the construction interfaces were modeled using the interface equivalent expansion method. The material properties were modeled using multi-output least-squares support vector regression optimized with the Jaya algorithm. A case study demonstrated that the established deformation health diagnosis model has good fitting and prediction ability. The model and methods proposed in this study provide a new idea for the behavior analysis and numerical simulation of mass composite structures similar to RCC dams.

Black-box modeling of ship maneuvering motion using multi-output least-squares support vector regression based on optimal mixed kernel function

Black-box modeling has been widely used to predict ship maneuvering motion. A novel black-box modeling approach using optimal mixed kernel multi-output least-squares support vector regression (MMLSSVR) based on the artificial hummingbird algorithm (AHA) is proposed in this paper. To improve the performance of the MMLSSVR model, a mixed kernel function that combines a radial basis function and a polynomial kernel function to capture both global and local properties is proposed. The optimal hyper-parameters of the mixed kernel function are determined by the AHA, which employs the linear combination of the mean square error and the determinant of error covariance as the objective function to balance between prediction accuracy and stability. The performance of the proposed method is evaluated using SR108 large container ship data. The experimental results demonstrate that the proposed MMLSSVR model achieves desirable prediction accuracy and strong generalizability for ship maneuvering motion modeling.

Simultaneous prediction of CO2, CO, and NOx emissions of biodiesel-hydrogen blend combustion in compression ignition engines by supervised machine learning tools

This study mathematically inspects the effects of engine speed/load and biodiesel–hydrogen fuel blend composition on CO2, CO, and NOx emissions. Since this modeling task requires a basic knowledge of mechanics, chemistry, and combustion, the literature suggests no approach to anticipate these emissions. This multi-input and multi-output (MIMO) problem is so complex that can only be simulated by special types of machine-learning tools. Consequently, this study utilizes the multi-output least-squares support vector regression (MLS-SVR) and two efficient artificial neural networks (cascade feedforward and multilayer perceptron) to handle this complicated MIMO regression. These intelligent models are constructed and tested using 307 × 3 experimental emissions related to the combustion of different biodiesel–hydrogen blends at various engine speeds and loads. Extensive numerical analyses and ranking tests are conducted to confirm that MLS-SVR is the best approach to simultaneously predict the actual CO2, CO, and NOx emissions with the mean absolute relative error of 2.75%, 7.00%, and 2.62%, and regression coefficient of 0.9966, 0.9947, and 0.9979, respectively. The applicability domain inspection approves that only seven emission measurements may be problematic samples and all other 300 samples are reliable.

Modeling the emission characteristics of the hydrogen-enriched natural gas engines by multi-output least-squares support vector regression: Comprehensive statistical and operating analyses

Modeling the emission characteristics of the hydrogen-enriched natural gas engines by multi-output least-squares support vector regression: Comprehensive statistical and operating analyses

Reconstruction of spatially continuous time-series land subsidence based on PS-InSAR and improved MLS-SVR in Beijing Plain area

ABSTRACT Beijing has undergone severe settlement in recent years. Persistent Scatterers Interferometric Synthetic Aperture Radar (PS-InSAR) technique has been widely used to derive time-series land deformation. However, existing studies have faced two challenges: (1) the nonlinear characteristics of time-series subsidence has not been fully investigated; (2) since PS points are normally distributed in urban areas with high building density, measurement gaps usually exist in nonurban areas. To address the challenges, we presented a new method to reconstruct spatially continuous time-series deformation. First, PS-InSAR was used to retrieve the deformation based on 135 scenes of Envisat ASAR and Radarsat-2 images from 2003 to 2020. Polynomial Curve Fitting (PCF) was then used to model nonlinear time-series deformation for the PS points. In the PS measurement gaps, Iterative Self-Organizing Data Analysis Technique (ISODATA) and Multi-output Least Squares Support Vector Regression (MLS-SVR) were used to estimate the PCF coefficients and then time-series deformation considering 40 features including thickness of the compressible layers, annual groundwater level, etc. The major results showed that (1) compared to linear, quadratic, and quartic models, cubic polynomial model generated better fit for the time-series deformation (R2 ≈0.99), suggesting obvious nonlinear temporal pattern of deformation; (2) the time-series deformation over measurement gaps reconstructed by ISODATA and MLS-SVR had satisfactory accuracy (R2 = 0.92, MAPE < 15%) and yielded higher accuracy (R2 = 0.947) than IDW (R2 = 0.687) and Ordinary Kriging (R2 = 0.688) interpolation methods. The reconstructed results maintain the nonlinear characteristics and ensure the high spatial resolution (120 m) of time-series deformation. Among the 40 predictor variables, ground water level datasets are the most influential predictors of time-series deformation.

A performance-enhanced DVL/SINS integrated navigation system based on data-driven approach

With the aid of Doppler velocity logger (DVL), strapdown inertial navigation systems (SINSs) can provide continuous and accurate navigation parameters for unmanned underwater vehicles (UUVs). However, owing to the complex underwater environment, partial DVL beams may fail to be reflected by the seafloor, resulting in DVL measurement outage. In this study, a novel data-driven approach enhancing DVL/SINS integrated navigation system is proposed to improve the robustness and accuracy of UUV navigation with limited DVL beams. First, to make full use of the available beam, the velocities of DVL beams are used to assist the SINS navigation instead of the 3D velocity of DVL. Subsequently, a virtual beam predictor based on multi-output least-squares support vector regression (MLS-SVR) is built to run in parallel with the DVL/SINS integrated system. Specifically, when the DVL operates normally, all four beams can be applied for integrated navigation and stored as the training dataset. Once partial beams are missing, the available beams and the outputs of SINS and pressure sensors can train the MLS-SVR model along with the corresponding missing beams in the training set. Subsequently, the missing beams can be predicted by the trained model for integrated navigation together with the available beams. UUV sea trials indicate that the proposed system can accurately predict unavailable beams and improve the positioning accuracy of the DVL/SINS integrated navigation system.

Parametric Model for Coaxial Cavity Filter with Combined KCCA and MLSSVR

Aiming at the problems of poor data effectiveness, low modeling accuracy, and weak generalization in the tuning process of microwave cavity filters, a parametric model for coaxial cavity filter using kernel canonical correlation analysis (KCCA) and multioutput least squares support vector regression (MLSSVR) is proposed in this study. First, the low-dimensional tuning data is mapped to the high-dimensional feature space by kernel canonical correlation analysis, and the nonlinear feature vectors are fused by the kernel function; second, the multioutput least squares support vector regression algorithm is used for parametric modeling to solve the problems of low accuracy and poor prediction performance; third, the support vector of the parameter model is optimized by the differential evolution whale algorithm (DWA) to improve the convergence and generalization ability of the model in actual tuning. Finally, the tuning experiments of two cavity filters with different topologies are carried out. The experimental results show that the proposed method has an obvious improvement in generalization performance and prediction accuracy compared with the traditional methods.

Quantification method of tubing defects based on machine learning algorithm and magnetic flux leakage signals.

Tubing is the pipeline that transports crude oil and natural gas from the oil and gas layer to the surface of the earth. Due to the harsh operating environment, the tubing will suffer from etch pits, scratches, cracks, perforations, and even direct fractures of different degrees of defective conditions. If tubing defects are not detected and quantified in a timely manner, the continued use of tubing will result in tubing leakage and failure. Magnetic flux leakage (MFL) testing as a nondestructive testing method enables the identification and quantitative analysis of defects in metal tubing. To improve the quantification accuracy of defects in the wellhead MFL testing of tubing defects during workover operations, this paper proposes a multi-output least-squares support vector regression machine (MLSSVR) model optimized based on the simulated annealing algorithm. The size of tubing defects can be quantified by establishing the mapping between the characteristic quantity of MFL signals and the defect size. The experimental results of MFL testing of tubing defects show that the root mean square error (RMSE) of the diameter of tubing defects of the simulated annealing algorithm optimized multi-output least-squares support vector regression (SA-MLSSVR) machine model proposed in this paper is 0.4562mm, and the RMSE of the depth of tubing defects is 0.1504mm. Compared with the non-optimized MLSSVR model, the overall RMSE of tubing defects is reduced by 36.48%. The SA-MLSSVR model only needs one-ninth of the time to achieve the same quantification accuracy as the particle swarm optimized multi-output least-squares support vector regression machine model.

Quality Prediction and Parameter Optimisation of Resistance Spot Welding Using Machine Learning

In a small sample welding test space, and to achieve online prediction and self-optimisation of process parameters for the resistance welding joint quality of power lithium battery packs, this paper proposes a welding quality prediction model. The model combines a chaos game optimisation algorithm (CGO) with the multi-output least-squares support vector regression machine (MLSSVR), and a multi-objective process parameter optimisation method based on a particle swarm algorithm. First, the MLSSVR model was constructed, and a hyperparameter optimisation strategy based on CGO was designed. Next, the welding quality was predicted using the CGO–MLSSVR prediction model. Finally, the particle swarm algorithm (PSO) was used to obtain the optimal welding process parameters. The experimental results show that the CGO–MLSSVR prediction model can effectively predict the positive and negative electrode nugget diameters, and tensile shear loads, with root mean square errors of 0.024, 0.039, and 5.379, respectively, which is better than similar methods. The average relative error in weld quality for the optimal welding process parameters is within 4%, and the proposed method has a good application value in the resistance spot welding of power lithium battery packs.

Intelligent candlestick forecast system for financial time-series analysis using metaheuristics-optimized multi-output machine learning

The effective prediction of stock market prices and trends is a critical topic in financial research for investors and stakeholders who wish to increase their return on investment. Motivated by highly unstable stock market targets and the continuous trading system that has been launched by Taiwan’s stock market, this work develops an intelligent candlestick forecast system (ICFS), which combines the advantages of three forecasting techniques — single-point, interval value, and trend forecasting. The ICFS is simple and intuitive, using a sliding-window metaheuristic-optimized multi-output least squares support vector regression hybrid model scheme to forecast simultaneously four types of price information that are represented on a candlestick chart. Three parameter-free metaheuristic algorithms, namely teaching-learning-based optimization (TLBO), symbiotic organisms search (SOS), and forensic-based investigation (FBI), are in turn used to optimize the hyperparameters of the multi-output least squares support vector regression (MLSSVR) model, to increase its accuracy and stability in predicting stock prices. Benchmark data from relevant studies are used to compare the forecast performances of previously developed and the three proposed hybrid models. Among all tested models, the FBI-MLSSVR greatly outperforms the others in forecasting Standard & Poor’s (S&P) and National Association of Securities Dealers Automated Quotations (NASDAQ) indexes, Taiwan’s exchange-traded funds, and major listed construction companies. A floating dollar-cost averaging strategy for maximizing returns while minimizing investment risks is proposed and compared with traditional dollar-cost averaging and the buy-and-hold strategy. The profitability of the predictions made using the proposed system is verified against actual stock market operations. The ICFS is packaged as a visualized expert system and independent application program for the convenience of users, who can intuitively run it and interpret its forecasts. Therefore, this study is highly practical by providing stakeholders with an easy-to-use and accurate predictive tool.

Auto-characterization of naturally fractured reservoirs drilled by horizontal well using multi-output least squares support vector regression

Pressure transient response (PTR) of horizontal well in naturally fractured reservoirs (NFR) has a particular characteristic shape. This PTR is often used to estimate parameters of NFRs and detect their wellbore and boundary regimes. Interporosity flow coefficient (λ) and storativity ratio (ω) are two important parameters of the NFR that often estimated by matching process on the PTR. Since the matching techniques’ results are not often unique, in this study, the multi-output least squares support vector regression (MLS-SVR) is employed for simultaneous estimation of λ and ω. A databank of 500 PTRs for horizontal wells in naturally fractured reservoirs is generated by the finite element method, converted to the pressure derivative (PD) curves, and then used to develop and evaluate this auto-characterization paradigm. The predictive accuracy of the model is checked and validated by both smooth and noisy PTRs. The proposed model predicts ω and λ with overall absolute average relative deviations (AARD) of 0.186% and 3.754%, respectively. The correlation coefficients (R2) of 1 and 0.99992 are obtained for the prediction of ω and λ, respectively. The Leverage outlier detection technique justified that only less than 6% of the predictions are within the suspect region. This MLS-SVR model can be simply integrated with commercial pressure transient analysis (PTA) packages for accurate prediction of ω and λ even from the noisy PTRs.

Direction of Arrival Based on the Multioutput Least Squares Support Vector Regression Model

The interferometer is a widely used direction-finding system with high precision. When there are comprehensive disturbances in the direction-finding system, some scholars have proposed corresponding correction algorithms, but most of them require hypothesis based on the geometric position of the array. The method of using machine learning that has attracted much attention recently is data driven, which can be independent of these assumptions. We propose a direction-finding method for the interferometer by using multioutput least squares support vector regression (MLSSVR) model. The application of this method includes the following: the construction of MLSSVR model training data, training and construction of the MLSSVR model, and the estimation of direction of arrival. Finally, the method is verified through numerical simulation. When there are comprehensive deviations in the system, the direction-finding accuracy can be effectively improved.

IDE-MLSSVR-Based Back Analysis Method for Multiple Mechanical Parameters of Concrete Dams

AbstractA back analysis method based on multioutput least-squares support vector regression machine (MLSSVR) and improved differential evolution algorithm (IDE) is proposed to estimate multiple mec...

A metaheuristic-optimized multi-output model for predicting multiple properties of pervious concrete

Pervious concrete can purify water, mitigate storm water runoff and reduce the urban heat island effect due to its larger porosity. However, its highly porous inner structure causes a lower compressive strength in comparison with normal concrete. Therefore it is vital to accurately predict the two basic parameters: permeability coefficient (PC) and uniaxial compressive strength (UCS) before field application to reduce time and cost of a construction project. As traditional mathematical models cannot model the highly nonlinear relationships between PC (or UCS) and its constituents, this study addresses this problem by applying a hybrid artificial intelligence model: multi-output least squares support vector regression (MOLSSVR). This model can also improve the prediction accuracy by utilizing the relationship between the two outputs: PC and UCS. In addition, the hyperparameters of MOLSSVR are tuned by a beetle-antennae search (BAS) algorithm which is modified by incorporating self-adaptive inertia weight and Levy flight. To train the proposed model, a large number of pervious concrete samples with different mixture proportions were prepared in laboratory. The results show that the searching efficiency of the modified BAS is significantly higher than that of BAS. The proposed hybrid model achieves better prediction accuracy than other models in the literature. This method can be used to address other multi-output problems in civil engineering.

Interval Forecasting of Financial Time Series by Accelerated Particle Swarm-Optimized Multi-Output Machine Learning System

By providing a range of values rather than a point estimate, accurate interval forecasting is critical to the success of investment decisions in exchange rate markets. This work proposes a sliding-window metaheuristic optimization for interval-valued time series forecasting using multi-output least squares support vector regression (MLSSVR). The hyperparameters in MLSSVR are finetuned using an accelerated particle swarm optimization algorithm to yield the best predictions and fastest convergence. The proposed system has a graphical user interface that is developed in a computing environment and functions as a stand-alone application. The system is validated using stock prices as well as exchange rates and outputs are compared with published results. Finally, the proposed interval time series prediction method is tested in two case studies; one involves the daily Australian dollar and Japanese yen rates (AUD/JPY) and the other involves US dollar and Canadian dollar rates (USD/CAD). The proposed model is promising for interval time series forecasting.

MULTIVARIATE CUSUM CONTROL CHART BASED ON THE RESIDUALS OF MULTIOUTPUT LEAST SQUARES SVR FOR MONITORING WATER QUALITY

Monitoring serially dependent processes using conventional control charts yields a high false alarm rate. Multioutput Least Squares Support Vector Regression (MLS-SVR) has the capability to encompass the cross-relatedness between output variables by learning multivariate output variables simultaneously. This research aims to develop a Multivariate Cumulative Sum (MCUSUM) control chart based on the residual obtained from the MLS-SVR model for monitoring autocorrelated data. The inputs of the MLS-SVR are selected using the significant lag of a partial autocorrelation function. The proposed control chart is applied to monitor water quality data and it can detect the assignable causes in those data caused by a broken pipeline.

On-line monitoring of multiple component parameters during ethanol fermentation by near-infrared spectroscopy

The quantity of biomass, glucose concentration and ethanol concentration are important parameters in ethanol fermentation. Traditional methods are usually based on samples for off-line measurement, which not only requires multiple instruments for test and analysis but also consumes notable time and effort, and therefore is inconvenient for real-time process control and optimization. In this study, an in-situ detection method based on the near-infrared (NIR) spectroscopy is proposed for measuring the above process parameters in real time. The in-situ measurement is carried out by using an immersion type NIR spectroscopy. A multi-output prediction model for simultaneously estimating the quantity of glucose, biomass and ethanol is established based on a multi-output least-squares support vector regression algorithm. The experimental results show that the proposed method can precisely measure the quantity of glucose, biomass and ethanol during the ethanol fermentation process. Compared to the existing partial-least-squares method for modeling and prediction of individual components, the proposed method could evidently improve the measurement accuracy and reliability.

Estimation of the depth of anesthesia by using a multioutput least-square support vector regression

Today, most surgeries are performed under general anesthesia where one of the most growing methods for anesthesia depth monitoring is using electroencephalogram (EEG). The bispectral index (BIS) is the most commonly used parameter for anesthesia depth monitoring using EEG, the validity of which is still to be studied before being accepted as a routine method by clinicians. This paper proposes a new technique for detecting the depth of anesthesia by means of EEG, which is based on multioutput least-squares support vector regression (MLS-SVR), which provides the probability that the patient is in the four different possible anesthesia states. In this study, EEG signals were recorded from 20 patients who were anesthetized in the operation room. Twelve linear and nonlinear EEG features were then extracted every 10 s from the EEG signals to form the feature vector. The features were then classified by the MLS-SVR classifier and the results were compared with those of the BIS, where no significant differences were observed (P > 0.05). Due to using the MLS-SVR classifier, which replaces quadratic equations by linear equations, the proposed method shows a higher accuracy compared to the other previously reported methods.

An efficient gradient-based model selection algorithm for multi-output least-squares support vector regression machines

Multi-output least-squares support vector regression machines (MLS-SVR) is proposed by Xu et al. [29] to handle multi-output regression problems. However, the prohibitive cost of model selection severely hinders MLS-SVR’s application. In this paper, an efficient gradient-based model selection algorithm for MLS-SVR is proposed. Firstly, a new training algorithm for MLS-SVR is developed, which allows one to obtain the solution vector for each output independently by dealing with matrices of much lower order. Based on the new training algorithm, a new leave-one-out error estimate is derived through virtual leave-one-out cross-validation. The model selection criterion is based on the new leave-one-out error estimate and its derivatives with respect to the hyper-parameters are also derived analytically. Both the model selection criterion and its partial derivatives can be obtained straightway once a training process ended. Finally, the hyper-parameters corresponding to the lowest model selection criterion is obtained through gradient decent method. The effectiveness and generalization performance of the proposed algorithm are validated through experiments on several multi-output datasets. Experiment results show that the proposed algorithm can save computational time dramatically without losing accuracy.