Least Squares Support Vector Machine Regression Research Articles

Quantification method of tubing defects based on machine learning algorithm and magnetic flux leakage signals.

Tubing is the pipeline that transports crude oil and natural gas from the oil and gas layer to the surface of the earth. Due to the harsh operating environment, the tubing will suffer from etch pits, scratches, cracks, perforations, and even direct fractures of different degrees of defective conditions. If tubing defects are not detected and quantified in a timely manner, the continued use of tubing will result in tubing leakage and failure. Magnetic flux leakage (MFL) testing as a nondestructive testing method enables the identification and quantitative analysis of defects in metal tubing. To improve the quantification accuracy of defects in the wellhead MFL testing of tubing defects during workover operations, this paper proposes a multi-output least-squares support vector regression machine (MLSSVR) model optimized based on the simulated annealing algorithm. The size of tubing defects can be quantified by establishing the mapping between the characteristic quantity of MFL signals and the defect size. The experimental results of MFL testing of tubing defects show that the root mean square error (RMSE) of the diameter of tubing defects of the simulated annealing algorithm optimized multi-output least-squares support vector regression (SA-MLSSVR) machine model proposed in this paper is 0.4562mm, and the RMSE of the depth of tubing defects is 0.1504mm. Compared with the non-optimized MLSSVR model, the overall RMSE of tubing defects is reduced by 36.48%. The SA-MLSSVR model only needs one-ninth of the time to achieve the same quantification accuracy as the particle swarm optimized multi-output least-squares support vector regression machine model.

Quality Prediction and Parameter Optimisation of Resistance Spot Welding Using Machine Learning

In a small sample welding test space, and to achieve online prediction and self-optimisation of process parameters for the resistance welding joint quality of power lithium battery packs, this paper proposes a welding quality prediction model. The model combines a chaos game optimisation algorithm (CGO) with the multi-output least-squares support vector regression machine (MLSSVR), and a multi-objective process parameter optimisation method based on a particle swarm algorithm. First, the MLSSVR model was constructed, and a hyperparameter optimisation strategy based on CGO was designed. Next, the welding quality was predicted using the CGO–MLSSVR prediction model. Finally, the particle swarm algorithm (PSO) was used to obtain the optimal welding process parameters. The experimental results show that the CGO–MLSSVR prediction model can effectively predict the positive and negative electrode nugget diameters, and tensile shear loads, with root mean square errors of 0.024, 0.039, and 5.379, respectively, which is better than similar methods. The average relative error in weld quality for the optimal welding process parameters is within 4%, and the proposed method has a good application value in the resistance spot welding of power lithium battery packs.

Aberrant brain gray matter and functional networks topology in end stage renal disease patients undergoing maintenance hemodialysis with cognitive impairment.

PurposeTo characterize the topological properties of gray matter (GM) and functional networks in end-stage renal disease (ESRD) patients undergoing maintenance hemodialysis to provide insights into the underlying mechanisms of cognitive impairment.Materials and methodsIn total, 45 patients and 37 healthy controls were prospectively enrolled in this study. All subjects completed resting-state functional magnetic resonance imaging (rs-fMRI) and diffusion kurtosis imaging (DKI) examinations and a Montreal cognitive assessment scale (MoCA) test. Differences in the properties of GM and functional networks were analyzed, and the relationship between brain properties and MoCA scores was assessed. Cognitive function was predicted based on functional networks by applying the least squares support vector regression machine (LSSVRM) and the whale optimization algorithm (WOA).ResultsWe observed disrupted topological organizations of both functional and GM networks in ESRD patients, as indicated by significantly decreased global measures. Specifically, ESRD patients had impaired nodal efficiency and degree centrality, predominantly within the default mode network, limbic system, frontal lobe, temporal lobe, and occipital lobe. Interestingly, the involved regions were distributed laterally. Furthermore, the MoCA scores significantly correlated with decreased standardized clustering coefficient (γ), standardized characteristic path length (λ), and nodal efficiency of the right insula and the right superior temporal gyrus. Finally, optimized LSSVRM could predict the cognitive scores of ESRD patients with great accuracy.ConclusionDisruption of brain networks may account for the progression of cognitive dysfunction in ESRD patients. Implementation of prediction models based on neuroimaging metrics may provide more objective information to promote early diagnosis and intervention.

Citrus Huanglongbing detection and semi-quantification of the carbohydrate concentration based on micro-FTIR spectroscopy.

Citrus Huanglongbing (HLB) is nowadays one of the most fatal citrus diseases worldwide. Once the citrus tree is infected by the HLB disease, the biochemistry of the phloem region in midribs would change. In order to investigate the carbohydrate changes in phloem region of citrus midrib, the semi-quantification models were established to predict the carbohydrate concentration in it based on Fourier transform infrared microscopy (micro-FTIR) spectroscopy coupled with chemometrics. Healthy, asymptomatic-HLB, symptomatic-HLB, and nutrient-deficient citrus midribs were collected in this study. The results showed that the intensity of the characteristic peak varied with the carbohydrate (starch and soluble sugar) concentration in citrus midrib, especially at the fingerprint regions of 1175-900cm-1, 1500-1175cm-1, and 1800-1500cm-1. Furthermore, semi-quantitative prediction models of starch and soluble sugar were established using the full micro-FTIR spectra and selected characteristic wavebands. The least squares support vector machine regression (LS-SVR) model combined with the random frog (RF) algorithm achieved the best prediction result with the determination coefficient of prediction ([Formula: see text]) of 0.85, the root mean square error of prediction (RMSEP) of 0.36%, residual predictive deviation (RPD) of 2.54, and [Formula: see text] of 0.87, RMSEP of 0.37%, RPD of 2.76, for starch and soluble sugar concentration prediction, respectively. In addition, multi-layer perceptron (MLP) classification models were established to identify HLB disease, achieving the overall classification accuracy of 94% and 87%, based on the full-range spectra and the optimal wavenumbers selected by the random frog (RF) algorithm, respectively. The results demonstrated that micro-FTIR spectroscopy can be a valuable tool for the prediction of carbohydrate concentration in citrus midribs and the detection of HLB disease, which would provide useful guidelines to detect citrus HLB disease.

Integrated Prediction Framework for Clinical Scores of Cognitive Functions in ESRD Patients.

The clinical scores are applied to determine the stage of cognitive function in patients with end-stage renal disease (ESRD). However, accurate clinical scores are hard to come by. This paper proposed an integrated prediction framework with GPLWLSV to predict clinical scores of cognitive functions in ESRD patients. GPLWLSV incorporated three parts, graph theoretic algorithm (GTA) and principal component analysis (PCA), whale optimization algorithm with Levy flight (LWOA), and least squares support vector regression machine (LSSVRM). GTA was adopted to extract features from the brain functional networks in ESRD patients, while PCA was used to select features. LSSVRM was built to explore the relationship between the selected features and the clinical scores of ESRD patients. Whale optimization algorithm (WOA) was introduced to select better parameters of the kernel function in LSSVRM; it aims to improve the exploration competence of LSSVRM. Levy flight was used to optimize the ability to jump out of local optima in WOA and improve the convergence of coefficient vectors in WOA, which lead to an increase in the generalization ability and convergence speed of WOA. The results validated that the prediction accuracy of GPLWLSV was higher than that of several comparable frameworks, such as GPSV, GPLSV, and GPWLSV. In particular, the average of root mean square error (RMSE), mean absolute error (MAE), and mean absolute percentage error (MAPE) between the predicted scores and the actual scores of ESRD patients was 2.40, 2.06, and 9.83%, respectively. The proposed framework not only can predict the clinical scores more accurately but also can capture imaging markers associated with decline of cognitive function. It helps to understand the potential relationship between structural changes in the brain and cognitive function of ESRD patients.

Brain Micro-Structural and Functional Alterations for Cognitive Function Prediction in the End-Stage Renal Disease Patients Undergoing Maintenance Hemodialysis

The goal of this study was to investigate the relationship between altered brain micro-structure and function, and cognitive function in patients with end-stage renal disease (ESRD) undergoing maintenance hemodialysis. Specially, diffusion kurtosis imaging (DKI), the resting-state functional connectivity (FC) algorithm, and the least squares support vector regression machine (LSSVRM) were utilized to conduct our study. A total of 50 patients and 36 matched healthy controls were prospectively enrolled in our study. All subjects completed the Montreal cognitive assessment scale (MoCA) test. DKI and resting-state functional magnetic resonance imaging were measured. Relationship between DKI parameters, FC, and MoCA scores was evaluated. LSSVRM combined with the whale optimization algorithm (WOA) was used to predict cognitive function scores. In ESRD patients, altered DKI metrics were identified in 12 brain regions. Furthermore, we observed changes in FC values based on regions of interest (ROIs) in nine brain regions, involved in default mode network (DMN), frontoparietal network (FPN), and the limbic system. Significant correlations among DKI values, FC values, and MoCA scores were found. To some extent, altered FC showed significant correlations with changed DKI parameters. Furthermore, optimized prediction models were applied to more accurately predict the cognitive function associated with ESRD patients. Micro-structural and functional brain changes were found in ESRD patients, which may account for the onset of cognitive impairment in affected patients. These quantitative parameters combined with our optimized prediction model may be helpful to establish more reliable imaging markers to detect and monitor cognitive impairment associated with ESRD.

Determination of the Soluble Solids Content in Korla Fragrant Pears Based on Visible and Near-Infrared Spectroscopy Combined With Model Analysis and Variable Selection.

The non-destructive detection of soluble solids content (SSC) in fruit by near-infrared (NIR) spectroscopy has a good application prospect. At present, the application of portable devices is more common. The construction of an accurate and stable prediction model is the key for the successful application of the device. In this study, the visible and near-infrared (Vis/NIR) spectra of Korla fragrant pears were collected by a commercial portable measurement device. Different pretreatment methods were used to preprocess the raw spectra, and the partial least squares (PLS) model was constructed to predict the SSC of pears for the determination of the appropriate pretreatment method. Subsequently, PLS and least squares support vector machine (LS-SVM) models were constructed based on the preprocessed full spectra. A new combination (BOSS-SPA) of bootstrapping soft shrinkage (BOSS) and successive projections algorithm (SPA) was used for variable selection. For comparison, single BOSS and SPA were also used for variable selection. Finally, three types of models, namely, PLS, LS-SVM, and multiple linear regression (MLR), were constructed based on different input variables. Comparing the prediction performance of all models, it showed that the BOSS-SPA-PLS model based on 17 variables obtained the best SSC assessment ability with rp of 0.94 and RMSEP of 0.27 °Brix. The overall result indicated that portable measurement with Vis/NIR spectroscopy can be used for the detection of SSC in Korla fragrant pears.

Compressed Machine Learning-Based Inverse Model for Design Optimization of Microwave Components

This article presents a new noniterative inverse modeling technique based on machine learning regression and its applications to microwave design optimization. The proposed inverse model accepts the high-dimensional S-parameters computed at many frequency points as the input and estimates the optimal geometrical/physical parameters of the microwave component as its output. The least-squares support vector machine regression is combined with the principal component analysis to simultaneously overcome both the high-dimensional input space and ill-posed challenges of the inverse modeling. We also propose a new empirical method to find the optimum number of principal components (i.e., the compression level) for each example in an automated way. This makes our proposed model general and easy to use compared with the existing data-driven inverse modeling techniques. The inverse model is trained by a set of scattering parameters computed via a 2-D/3-D solver for few configurations of the geometrical parameters. The feasibility and the accuracy of the proposed optimization scheme are investigated by comparing its predictions with the corresponding optimal configuration estimated via a commercial solver.

Compressed Complex-Valued Least Squares Support Vector Machine Regression for Modeling of the Frequency-Domain Responses of Electromagnetic Structures

This paper deals with the development of a Machine Learning (ML)-based regression for the construction of complex-valued surrogate models for the analysis of the frequency-domain responses of electromagnetic (EM) structures. The proposed approach relies on the combination of two-techniques: (i) the principal component analysis (PCA) and (ii) an unusual complex-valued formulation of the Least Squares Support Vector Machine (LS-SVM) regression. First, the training and test dataset is obtained from a set of parametric electromagnetic simulations. The spectra collected in the training set are compressed via the PCA by exploring the correlation among the available data. In the next step, the compressed dataset is used for the training of compact set of complex-valued surrogate models and their accuracy is evaluated on the test samples. The effectiveness and the performance of the complex-valued LS-SVM regression with three kernel functions are investigated on two application examples consisting of a serpentine delay structure with three parameters and a high-speed link with four parameters. Moreover, for the last example, the performance of the proposed approach is also compared with those provided by a real-valued multi-output feedforward Neural Network model.

GWLS: A Novel Model for Predicting Cognitive Function Scores in Patients With End-Stage Renal Disease.

The scores of the cognitive function of patients with end-stage renal disease (ESRD) are highly subjective, which tend to affect the results of clinical diagnosis. To overcome this issue, we proposed a novel model to explore the relationship between functional magnetic resonance imaging (fMRI) data and clinical scores, thereby predicting cognitive function scores of patients with ESRD. The model incorporated three parts, namely, graph theoretic algorithm (GTA), whale optimization algorithm (WOA), and least squares support vector regression machine (LSSVRM). It was called GTA-WOA-LSSVRM or GWLS for short. GTA was adopted to calculate the area under the curve (AUC) of topological parameters, which were extracted as the features from the functional networks of the brain. Then, the statistical method and Pearson correlation analysis were used to select the features. Finally, the LSSVRM was built according to the selected features to predict the cognitive function scores of patients with ESRD. Besides, WOA was introduced to optimize the parameters in the LSSVRM kernel function to improve the prediction accuracy. The results validated that the prediction accuracy obtained by GTA-WOA-LSSVRM was higher than several comparable models, such as GTA-SVRM, GTA-LSSVRM, and GTA-WOA-SVRM. In particular, the root mean square error (RMSE), mean absolute error (MAE), and mean absolute percentage error (MAPE) between the predicted scores and the actual scores of patients with ESRD were 0.92, 0.88, and 4.14%, respectively. The proposed method can more accurately predict the cognitive function scores of ESRD patients and thus helps to understand the pathophysiological mechanism of cognitive dysfunction associated with ESRD.

Spectral inversion model of the crushing rate of soybean under mechanized harvesting

Rapid and timely acquisition of the crushing rate can help in assessing the performance of combine harvesters, which is very important for agricultural production. The spectral reflectance of soybean provides an alternative method to the classical physical and chemical analysis of the crushing rate of soybean in laboratory. Therefore, hyperspectral imaging can be used to rapidly obtain the crushing rate of soybean. In this study, the hyperspectral method was employed, and the application of inter-correlation analysis was explored in the optimization and quantitative analysis of hyperspectral bands. The crushing rate of 130 soybean samples collected from a combine harvester was investigated through physical analysis in the laboratory. Subsequently, the raw hyperspectral reflectance of soybean samples was measured using a spectroradiometer equipped with a high intensity contact probe under darkroom conditions. Next, the raw spectral reflectance (REF) and the logarithmic reciprocal pretreatment spectrum data (LR) were analyzed and compared. The effective wavelengths were selected according to the results of the inter-correlation analysis. Regression models of the crushing rate with different indices were established using a least squares support vector machine (LS-SVM). The inversion results of the model were validated and compared with each other. The experimental results show that sensitive bands from REF are 1061, 1068, 1074, 1090, 2085, 2092, 2095, and 2103 nm. Sensitive bands from LR are 677, 1039, 1078, 1093, 1101, 1956, 2088, and 2107 nm. The results showed that REF was the optimal spectral index in the LS-SVM regression model (Rc2 was 0.939, and Rp2 was 0.915). The inter-correlation analysis method could not only support efficient selection of hyperspectral bands, but also retain the original sample information. The REF hyperspectral inversion model based on LS-SVM can realize rapid on-line monitoring of the performance (crushing rate) of grain combine harvesters in the future.

Natural disaster warning system for safe operation of a high-speed railway

Abstract In this paper, the least square support vector machine (LSSVM) is used to study the safety of a high-speed railway. According to the principle of LSSVM regression prediction, the parameters of the LSSVM are optimized to model the natural disaster early warning of safe operation of a high-speed railway, and the management measures and methods of high-speed railway safety operation under natural disasters are given. The relevant statistical data of China's high-speed railway are used for training and verification. The experimental results show that the LSSVM can well reflect the nonlinear relationship between the accident rate and the influencing factors, with high simulation accuracy and strong generalization ability, and can effectively predict the natural disasters in the safe operation of a high-speed railway. Moreover, the early warning system can improve the ability of safety operation evaluation and early warning of high-speed railway under natural disasters, realize the development goals of high-speed railway (safety, speed, economic, low-carbon and environmental protection) and provide a theoretical basis and technical support for improving the safety of a high-speed railway.

Quantitative analysis of fish meal freshness using an electronic nose combined with chemometric methods

Total volatile basic nitrogen (TVB-N) and acid value (AV) are the main indicators for characterizing fish meal freshness. To explore the feasibility of fish meal freshness detection based on an electronic nose system, a portable electronic nose measurement system was developed to detect the TVB-N and AV mass fraction in fish meal. The device is mainly composed of a gas acquisition and transmission module, a control and processing storage module with Raspberry Pi as the core, an ARPI600 data acquisition module, and a sensor array module. The system was used to collect the odour signals of fish meal samples with different TVB-N and AV mass fractions and the chemical test was conducted for the TVB-N and AV mass fractions in fish meal samples. The feature selection method of recursive feature elimination based on random forest (RFRFE) was used to select the sensor array features, and models of the TVB-N and AV mass fraction in fish meal were established by random forest regression (RFR) and least squares support vector machine (LSSVM) model based on the particle swarm optimization (PSO) algorithm, and the models were verified by a test sample. The experimental results show that the performance of the LSSVM regression model based on the RFRFE optimized sensor array data set is better than that of the traditional LSSVM model, The R2 and root mean square error (RMSE) between the predicted values and the real values for AV and TVB-N based on the test set are 0.8585, 0.3622 and 0.8789, 20.52, respectively. Therefore, the performance of feature selection using the RFRFE method is better. The results show that the TVB-N and AV of fish meal can be detected effectively by electronic nose technology.

Multiple Delay Identification in Long Interconnects via LS-SVM Regression

This work presents a novel approach for the accurate estimation of multiple time-delays from the frequency response of a distributed system. The proposed approach is based on a powerful and flexible machine learning technique, namely, the least-square support vector machine (LS-SVM). The LS-SVM regression is used to construct a metamodel of the transfer function describing a generic linear time-invariant system in a delayed-rational form. Specifically, after some manipulation the LS-SVM model precisely identifies the dominant propagation delays of the original system. The essential steps and critical criteria for the delay identification procedure are carefully discussed throughout the paper. Once the system delays have been identified, the rational part of the metamodel expansion is then obtained by means of a progressive application of the conventional vector fitting algorithm. Numerical examples are presented to illustrate the feasibility and performance of the proposed technique and to compare its performances with what is provided by state-of-the-art techniques. The results clearly highlight the capability of the proposed approach to identify the dominant delays in distributed systems, thus allowing to construct compact delayed rational models.

Bayesian Optimization of Hyperparameters in Kernel-Based Delay Rational Models

This paper presents an automatic procedure for the optimization of the hyperparameters of a delay rational model approximating the frequency-domain behavior of highspeed interconnects. The proposed model is built via a kernel-based regression, such as the Least-Square Support Vector Machine (LS-SVM), by considering an ad-hoc kernel with two hyperparameters related to the propagation delays introduced by the system. Such hyperparameters, along with the Tikhonov regularizer used by the LS-SVM regression, are carefully tuned via an automatic approach based on a k-fold cross-validation and Bayesian optimization. The feasibility of the effectiveness of the proposed modeling approach are investigated on a high-speed link.

Machine Learning for the Uncertainty Quantification of Power Networks

This letter addresses the uncertainty quantification of a power network and is based on surrogate models built via Machine Learning techniques. Specifically, the least-square support vector machine regression is combined with the principal component analysis to generate a compressed surrogate model capable of predicting all the nodal voltages of the network as a function of the uncertain electrical parameters of the transmission lines. The surrogate model is built from a limited number of system responses provided by the computational model. The power flow analysis of the benchmark IEEE-118 bus system with 250 parameters is considered as a test case. The performance of the proposed modeling strategy in terms of accuracy, efficiency and convergence, are assessed and compared with those of an alternative surrogate model based on a sparse implementation of the polynomial chaos expansion. The results of a Monte Carlo simulation are used as reference in the above comparison.

PERMANENT SYNCHRON MAGNET MOTOR SPEED OBSERVER BASED ON LEAST SQUARES SUPPORT VECTOR MACHINE REGRESSION

Motor speed control requires motor speed data as feedback from control actions. Motor speed data is usually obtained from the speed sensor. In this paper, the motor speed observer for permanent magnet synchronous motor is proposed to obtain motor speed data based on motor back emf voltage making it more economical without a speed sensor. The Speed observer is designed based on the Model Reference Adapative System (MRAS) with using Least Squares Support Vector Machine Regression (LSSVMR) algorithm for adaptation mechanism tools. The proposed speed observer is tested with varying motor speeds. The test results show that the MRAS-based motor speed observer using LSSVMR has successfully estimated the rotation speed of the permanent magnet synchronous motor based on the back emf motor voltage. It can be seen from the maximum error of the motor speed, ie only 3.7 rpm at transient conditions and close to zero at steady state

IDE-MLSSVR-Based Back Analysis Method for Multiple Mechanical Parameters of Concrete Dams

AbstractA back analysis method based on multioutput least-squares support vector regression machine (MLSSVR) and improved differential evolution algorithm (IDE) is proposed to estimate multiple mec...

A computer-assisted soil texture analysis using digitally scanned images

A computer-assisted soil texture analysis is presented using digitally scanned soil images of 177 soil samples collected from different regions of Brazil. Soil digital images were correlated to texture results determined by the standard pipette method using three multivariate methods: successive projections algorithm combined with multivariate linear regression (SPA-MLR), partial least-squares regression (PLSR), and least-squares support vector machine regression (LSSVMR). Sand and clay particle size were better estimated using LSSVMR presenting correlations above 90%. Following soil sample particle size content estimates, soil texture classes were also estimated achieving 90.6% accuracy. The proposed method using digital images is fast, cheap and has low environmental impact when compared to the standard method.

A deep learning based regression method on hyperspectral data for rapid prediction of cadmium residue in lettuce leaves

In order to effectively realize the spectral detection of heavy metal content, a deep learning method which consisted of stacked auto-encoders (SAE) and partial least squares support vector machine regression (LSSVR) is proposed to obtain depth features and establish cadmium (Cd) detection model. The Vis-NIR hyperspectral images of 1120 lettuce leaf samples were obtained and the whole region of lettuce leaf sample spectral data was collected and preprocessed with different spectral pre-treatment methods. Successive projections algorithm (SPA), partial least squares regression (PLSR) and SAE were used to acquire the optimum wavelength, respectively. Besides, the characteristic wavelengths were used to build partial least squares support vector machine regression (LSSVR) models. Furthermore, the best prediction performance for detecting Cd content in lettuce leaves was obtained by Savitzky-Golay combined with first derivative (SG-1st) pre-processing method, with Rp2 of 0.9487, RMSEP of 0.01049 ​mg/kg and RPD of 3.330 using SAE-LSSVR method. The results of this study indicated that deep learning method coupled with hyperspectral imaging technique has great potential for detecting heavy metal content in lettuce leaves.