Multi-output Gaussian Process Regression Research Articles

System fragility analysis of highway bridge using multi-output Gaussian process regression surrogate model

This study presents a multi-output Gaussian process regression (GPR) surrogate model for seismic-fragility analysis of bridge structural systems. Multi-output GPR can model the correlations among multiple outputs, accuracy and stability are achieved with fewer training data, which reduces the computational cost of fragility analysis. Furthermore, the explainability of the constructed surrogate model is implemented by adopting an automatic relevance-determination (ARD) kernel in the GPR. The estimated hyperparameters can provide the contribution of the uncertainty of each input parameter to the outputs. The fragility analysis using the multi-output GPR surrogate model was verified by applying it to a seismic isolation highway bridge with multiple spans and a curved geometry. The effectiveness of the multi-output GPR was demonstrated by the construction of an accurate and stable surrogate model with 46 inputs and 28 outputs. The relative contributions of the uncertainties to the structural properties and input earthquake loads could also be understood. The fragility curves, at both the component and system levels, were appropriately obtained using a sufficient number of samples in a Monte Carlo calculation. Furthermore, the failure modes were evaluated, identifying which structural components contributed to the system failure. This enabled discussions on structural system failures from the viewpoint of the structural dynamic characteristics of the bridge and earthquake-load properties.

A probabilistic machine learning framework for stiffness tensor estimation of carbon composite laminate

The potential uses of carbon composite material are vast, particularly in the civil, mechanical and aero-structures. Nonetheless, the practical utilization of carbon composite faces challenges due to complex experimental methods and imprecise algorithms for inverting material properties. Characterization of full set of stiffness tensor for an anisotropic material in particular orthotropic material can be considered as a complex non-linear inversion problem. The inversion process requires a robust optimization algorithm and forward models. The complexity of this inversion process impedes the practical utility in large-scale automation and in real-time structural health monitoring (SHM) application. To cater this, in this work an advanced probabilistic machine learning framework based on multi-output Gaussian process regression (moGPR) is proposed as an inversion algorithm. The inversion algorithm proposed is based on probabilistic framework that utilises the dispersion of Lamb waves as an input for optimal estimation of stiffness tensor. Experimental measurements of full wavefield data of propagating waves are conducted by scanning laser Doppler vibrometer. Further, an optimal training dataset is generated by employing a forward model based on stiffness matrix method (SMM). In the presence of measurement uncertainties, the effectiveness of the optimal prediction algorithm is showcased through a comparison between the estimated parameter and its ground truth. This comparison reveals that the proposed inversion algorithm is both efficient and robust, delivering satisfactory performance even in the presence of substantial measurement uncertainties.

Evaluation of machine tool dynamics in dry gear hobbing considering time-varying thermal effects

Effective stability analysis strongly depends on accurately predicting machine tool dynamics. However, the nonlinear thermal effects during the machining operations present a significant challenge to accurately modeling the machine tool dynamics. To this end, the evolution of the machine tool dynamics under the thermal growth during the dry gear hobbing process is investigated, and a rapid evaluation model of the temperature-dependent dynamics is developed. Firstly, a preliminary test was conducted to explore thermal effects on the machine tool dynamics. Then, the free vibrations at the spindle nose during the hobbing operations were filtered, and the corresponding modal parameters were estimated. Subsequently, the influence of temperature rise on the modal parameters is analyzed. The results show that the first- and third-order natural frequencies increase by 142 and 95 Hz, respectively, and the damping ratios for all modes are reduced with the machine tool temperature rise. Finally, the relationship between the feature temperatures and modal parameters is described using the multi-output Gaussian Process regression (MOGPR) model. The overall prediction accuracy is 95.6 %, and the relative errors in the test data set are within 10 %. The findings can provide an essential basis for the machining stability analysis of dry gear hobbing.

Probabilistic impact localization in composites using wavelet scattering transform and multi-output Gaussian process regression

Data-driven machine-learning models offer considerable promise for acoustic source localization. However, many existing models rely on training data that correlates time-of-flight (TOF) measurements with source locations, yet they struggle to handle the complexities arising from nonlinear wave propagation in materials with varying properties. Furthermore, these models overlook the noise and uncertainties inherent in real-world experiments when predicting outputs. This paper aims to bridge a gap in impact localization for such structures, particularly focusing on scenarios involving noisy field measurements. This study proposes a framework based on probabilistic machine learning to identify impact locations, utilizing wavelet scattering transform (WST) and Multi-Output Gaussian Process Regression (moGPR). WST extracts informative features from Lamb waves, capturing relevant signatures for training the probabilistic machine learning model, while moGPR estimates correlated impact location coordinates (x, y) while accounting for inherent uncertainties in the data. To assess the proposed method’s performance in handling measurement uncertainties, an experiment was conducted using a CFRP composite panel instrumented with a sparse array of piezoelectric transducers. The results demonstrate that the probabilistic framework effectively addresses measurement uncertainties, enabling reliable source location estimation with confidence intervals and providing valuable insights for decision-making.

Data-driven uncertainty quantification and sensitivity studies in free vibration behavior of bio-inspired helicoidal laminated composite cylindrical shells

The uncertainty in the ply orientation of bio-inspired helicoidal laminated composite cylindrical shells is quantified. The frequencies obtained from the higher-order shear deformation theory are trained to formulate a surrogate model in the multi-output Gaussian Process Regression algorithm framework, and the stochastic analysis is then carried out using the Monte Carlo simulation method. For simply supported and clamped shells, uncertainties in the top and bottom-most layers widely affect the free vibration behavior of the shells compared to the inner-most or central layers, whereas for cantilever shells, opposite trend is observed. The inclusion of noise widely affects the free vibration behavior.

Prediction of pose- and position-dependent tool-tip dynamics in high-speed dry gear hobbing

High-speed dry gear hobbing is an advanced and efficient machining technology for automotive gears. Frequent changes in the hob mounting angle (tilt angle) and shifting position result in variations in the tool-tip frequency response function (FRF), which inevitably modify the hobbing stability boundary. To construct an accurate stability boundary for high-speed dry gear hobbing, the primary objective is to predict the tool-tip FRF related to different combinations of hob tilt angles and shifting positions, that is, pose- and position-dependent tool-tip FRF. This study proposes a novel tool-tip FRF prediction strategy that combines multi-output Gaussian Process regression (MOGPR) and Receptance Coupling Substructure Analysis (RCSA). First, the FRFs at the tips of the hob-mounting base were estimated through a series of impact tests under a limited combination of hob tilt angles and shifting positions, and the corresponding modal parameters were identified. The MOGPR model was then leveraged to reveal the mapping relationships between the modal parameters, hob tilt angles and shifting positions. Based on this, the hob-mounting base pose- and position-dependent FRFs can be obtained using the modal fitting technique. Subsequently, the pose-and position-dependent tool-tip FRFs were obtained by coupling the receptances of the hob-toolbar assembly with the fitted hob-mounting base FRFs through the RCSA. Finally, a series of impact tests were conducted to verify the effectiveness of the proposed method. The prediction of pose- and position-dependent tool-tip dynamics can provide a foundation for accurately modeling hobbing stability.

Latent Stochastic Differential Equations for Modeling Quasar Variability and Inferring Black Hole Properties

Quasars are bright and unobscured active galactic nuclei (AGN) thought to be powered by the accretion of matter around supermassive black holes at the centers of galaxies. The temporal variability of a quasar’s brightness contains valuable information about its physical properties. The UV/optical variability is thought to be a stochastic process, often represented as a damped random walk described by a stochastic differential equation (SDE). Upcoming wide-field telescopes such as the Rubin Observatory Legacy Survey of Space and Time (LSST) are expected to observe tens of millions of AGN in multiple filters over a ten year period, so there is a need for efficient and automated modeling techniques that can handle the large volume of data. Latent SDEs are machine learning models well suited for modeling quasar variability, as they can explicitly capture the underlying stochastic dynamics. In this work, we adapt latent SDEs to jointly reconstruct multivariate quasar light curves and infer their physical properties such as the black hole mass, inclination angle, and temperature slope. Our model is trained on realistic simulations of LSST ten year quasar light curves, and we demonstrate its ability to reconstruct quasar light curves even in the presence of long seasonal gaps and irregular sampling across different bands, outperforming a multioutput Gaussian process regression baseline. Our method has the potential to provide a deeper understanding of the physical properties of quasars and is applicable to a wide range of other multivariate time series with missing data and irregular sampling.

Prediction of the Modal Parameters of The Spindle System in High-speed Dry Hobbing Machine with Thermal Effects

High-speed dry hobbing is an advanced green machining technology for gears. However, thermal issues in the machine tools become serious under the high cutting speed and dry cutting conditions, which result in a nonlinear evolution of the machine tool dynamics, and finally affect the hobbing stability. In view of this, introducing the hob spindle system as the object, this paper intends to investigate the influence of time-varying temperatures on the modal parameters based on the modal tests. Then, a temperature-dependent modal parameters prediction model based on the multi-output Gaussian process regression (MOGPR) is proposed. The experimental results indicated that the modal parameters of the spindle system vary obviously with changing thermal states. The natural frequencies for the first and third modes increased with the temperature rise, while the variation of the natural frequency for the second mode was insignificant. The damping ratios for the first three modes showed a decreasing trend with the temperature rise. The constructed MOGPR model has the capability to effectively predict the modal parameters with the time-varying thermal effects, which can provide a support to build the temperature-dependent hobbing stability boundaries.

Prediction and correlations estimation of seismic capacities of pier columns: Extended Gaussian process regression models

Assessing the seismic capacity of pier columns is a crucial element in the performance-based seismic design of bridges. Such assessment necessitates a probabilistic approach to accurately determine the marginal probability distributions of seismic capacities and to characterize the dependencies among these variables. In response to this need, this paper employs Multi-Output Gaussian Process Regression (MO-GPR), a probabilistic machine learning method, to jointly model the multiple seismic capacities of pier columns. We initially introduce a probabilistic seismic capacity model that utilizes MO-GPR for pier columns and validate its predictive accuracy in comparison to Bayesian linear regression and existing empirical methods. Subsequently, the methodology is augmented by the integration of hierarchical modeling within the MO-GPR framework, resulting in a Multi-Output Hierarchical Gaussian Process Regression (MO-HGPR) model that effectively estimates intraclass correlation coefficients for specific types of datasets. It is postulated that these correlation coefficients also reflect correlations associated with multiple components of the real structure. This study employs MO-HGPR and MO-GPR separately to investigate the potential correlations of seismic capacities among pier columns and distinct limit states. The results demonstrate that the MO-GPR model exhibits superior prediction accuracy and more effectively portrays uncertainty compared to existing empirical models. Moreover, the correlations of seismic capacities among piers and limit states are both robust and significantly impact the seismic fragility of bridges. This finding highlights the essential nature of considering capacities correlations in seismic fragility or risk assessment processes.

Performance evaluation of supersonic flow for variable geometry radial ejector through CFD models based on DES-turbulence models, GPR machine learning, and MPA optimization

This study aims to derive valuable insights for utilizing computational fluid dynamics (CFD) based on reynolds-averaged navier–stokes (RANS) and detached eddy simulation (DES) turbulence models (TMs) to analyze a specific radial ejector configuration known as the variable geometry radial ejector (VGRE). The VGRE features the primary nozzle and ejector duct plates with adjustable disk-like surfaces, allowing for changes in the nozzle and ejector throat areas within a single ejector. Extensive numerical investigations of the VGRE are conducted by systematically validating CFD models with experimental datasets and subsequently using the most appropriate TM to design a new radial ejector. The study reveals that the DES SST k-ω turbulence model achieves the closest agreement with experimental data, with an average entrainment ratio (ω) discrepancy of only 5 %. However, there are challenges in accurately predicting the critical compression ratio (rc*), especially under varying conditions. Based on the CFD results, the original VGRE exhibited ω values ranging from 0.16 to 0.61, rc* values between 1.5 and 3.1, and ejector efficiency (η) values between 7 % and 17 % at expansion ratio (re) values ranging from 89 to 150 for different nozzle throat separations (d) and different duct throat separations (D). Furthermore, this study presents a comprehensive investigation into predicting and optimizing ω, rc*, and η parameters using a multi-output gaussian process regression (GPR) model and a marine predators algorithm (MPA) approach. The multi-output GPR model was constructed to predict the relationships between boundary conditions (primary pressure (Pprimary) and secondary pressure (Psecondary)), geometric parameters (d and D), and the response variables (ω, rc*, and η). The model evaluation employed a 5-fold cross-validation technique to assess predictive performance, demonstrating strong predictive accuracy with low root mean square error (RMSE) and high coefficient of determination (R2) values. The optimization results revealed that the highest achieved values were ω = 0.303, rc* = 2.678, and η = 0.156, corresponding to specific parameter settings (Pprimary = 160 kPa, Psecondary = 1.8 kPa, d = 0.6 mm, and D = 2.3 mm). This study demonstrates the effectiveness of the multi-output GPR model for accurate prediction and the multi-objective MPA optimization approach for identifying optimal input parameters to maximize entrainment ratio, critical compression ratio, and ejector efficiency in fluid dynamics systems.

Plane Cascade Aerodynamic Performance Prediction Based on Metric Learning for Multi-Output Gaussian Process Regression

Multi-output Gaussian process regression measures the similarity between samples based on Euclidean distance and assigns the same weight to each feature. However, there are significant differences in the aerodynamic performance of plane cascades composed of symmetric and asymmetric blade shapes, and there are also significant differences between the geometry of the plane cascades formed by different blade shapes and the experimental working conditions. There are large differences in geometric and working condition parameters in the features, which makes it difficult to accurately measure the similarity between different samples when there are fewer samples. For this problem, a metric learning for the multi-output Gaussian process regression method (ML_MOGPR) for aerodynamic performance prediction of the plane cascade is proposed. It shares parameters between multiple output Gaussian distributions during training and measures the similarity between input samples in a new embedding space to reduce bias and improve overall prediction accuracy. For the analysis of ML_MOGPR prediction results, the overall prediction accuracy is significantly improved compared with multi-output Gaussian process regression (MOGPR), backpropagation neural network (BPNN), and multi-task learning neural network (MTLNN). The experimental results show that ML_MOGPR is effective in predicting the performance of the plane cascade, and it can quickly and accurately make a preliminary estimate of the aerodynamic performance and meet the performance parameter estimation accuracy requirements in the early stage.

Federated Multi-Output Gaussian Processes

Multi-output Gaussian process (MGP) regression plays an important role in the integrative analysis of different but interrelated systems/units. Existing MGP approaches assume that data from all units is collected and stored at a central location. This requires massive computing and storage power at the central location, induces significant communication traffic due to raw data exchange, and comprises privacy of units. However, recent advances in Internet of Things technologies, which have tremendously increased edge computing power, pose a significant opportunity to address such challenges. In this article, we propose FedMGP, a general federated analytics (FA) framework to learn an MGP in a decentralized manner that uses edge computing power to distribute model learning efforts. Specifically, we propose a hierarchical modeling approach where an MGP is built upon shared global latent functions. We then develop a variational inference FA algorithm that overcomes the need to share raw data. Instead, collaborative learning is achieved by only sharing global latent function statistics. Comprehensive simulation studies and a case study on battery degradation data highlight the superior predictive performance and versatility of FedMGP, achieved while distributing computing and storage demands, reducing communication burden, fostering privacy, and personalizing analysis.

Multi-scale adaptive multivariate state estimation fault detection enhancement for time-varying industrial system based on multi-output Gaussian process autoregression

Most chemical processes are time-varying and nonlinear due to load changes, product demand transitions or other causes, causing unsatisfactory performance of conventional non-parametric multivariate statistical fault detection method. In this work, a probabilistic multivariate state estimation fault detection method based on multi-output Gaussian process autoregression is first developed to overcome the linear limitation and noise information ignorance of multivariate state estimation technology-based fault detection methods. In multi-output Gaussian process regression framework, the Leave-One-Sample-Out strategy is developed to construct the dataset by leaving each historical sample from the memory matrix out as one training output in turn while regarding the corresponding remaining historical samples as one training input. Then, we design the multi-output Gaussian process autoregression model to estimate the current state of the system. Utilizing the estimation covariance information, a probabilistic index based on multivariate Kullback-Leibler divergence is developed for fault detection. Moreover, a multi-scale adaptive fault detection enhancement based on discrete multi-objective Archimedes optimization and just-in-time learning is further proposed to handle the time-varying problem. In this scheme, the memory matrix is batch-adaptively updated at monthly intervals based on discrete multi-objective Archimedes optimization to handle the slow change of system characteristics caused by equipment degradation. On the other hand, to cope with the rapid change of system characteristics caused by operation transition, the just-in-time learning is employed to select samples from the memory matrix that best match to the current system state for sample-adaptive updating. Finally, two industrial cases are used to verify the performance of the proposed method.

Self-Evolving Offset-Free Model Predictive Control with Model–Plant Mismatch for Dynamic Working-Point Change Tasks in Industrial Processes

This study proposes a self-evolving offset-free model predictive control (MPC) algorithm for dynamic working-point change (DWPC) tasks in industrial processes. The algorithm mitigates disturbance impacts caused by model–plant mismatch (MPM) and enhances the dynamic performance of MPC by locating sequences (scenarios) similar to the current operational scenario from historical DWPC tasks and using them for multistep-ahead disturbance prediction. First, a disturbance-augmented state–space model guarantees the basic offset-free control behavior of MPC with MPM. Next, to enhance the MPC performance, a direct multistep-ahead disturbance prediction approach is proposed by combining historically similar DWPC task scenarios. Specifically, a dynamic autoencoder is constructed to extract private features from process scenarios and locate similar scenarios from historical DWPC tasks. Based on the located scenarios, the multistep-ahead disturbance and its uncertainty are directly predicted through multioutput Gaussian process regression. Finally, the obtained disturbance results are incorporated into the MPC framework, which continuously enhances MPC performance in DWPC tasks. Two case studies demonstrate the effectiveness of the proposed MPC.

Dose–response prediction for in-vitro drug combination datasets: a probabilistic approach

In this paper we propose PIICM, a probabilistic framework for dose–response prediction in high-throughput drug combination datasets. PIICM utilizes a permutation invariant version of the intrinsic co-regionalization model for multi-output Gaussian process regression, to predict dose–response surfaces in untested drug combination experiments. Coupled with an observation model that incorporates experimental uncertainty, PIICM is able to learn from noisily observed cell-viability measurements in settings where the underlying dose–response experiments are of varying quality, utilize different experimental designs, and the resulting training dataset is sparsely observed. We show that the model can accurately predict dose–response in held out experiments, and the resulting function captures relevant features indicating synergistic interaction between drugs.

An integrated machine learning framework for effluent quality prediction in Sewage Treatment Units

ABSTRACT The influent and effluent data from wastewater treatment plants being highly correlated with multi-variable coupling and time-varying features may degrade the performance of conventional soft sensors over time. Adaptive strategies based on just-in-time learning (JIT), moving windows (MW), and time difference (TD) are used in this work to develop an adaptive soft sensor. Multi-output Gaussian-process regression (MGPR) is selected and hybrid methods such as TD JIT, MW TD, and JIT MW TD along with TD and MGPR methods are implemented. Data from the benchmark simulation model No.1, closed-loop architecture after applying PI controller, and real-time data from the Rithala Plant of Delhi are obtained. The improved error percentage is 15.03% for total phosphorus (open-loop) using the JIT TD method when compared with the MW TD method. Fair results are observed with JIT TD on real time data with a strong correlation between predicted and observed values, above 0.8 for any variable being estimated.

Control strategy of multi-stand work roll bending and shifting on the crown for UVC hot rolling mill based on MOGPR approach

In order to improve the crown control ability and high quality to meet the Profile Contour Flatness Control (PCFC) requirements of wide electrical steel in the hot strip mills (HSMs), the control strategy of multi-stand work roll bending forces and shifting stroke on the crown for the universal variable crown (UVC) mill is established based on the Multi-output Gaussian Process Regression (MOGPR) method. The UVC technology with wide crown control ability is proposed for the upstream stands, as effective and critical crown control for upstream stands, of the HSMs. The prediction model of UVC work roll shifting was developed by integrating the MOGPR method and the production process data, and it has high prediction accuracy and adaptability compared with BPNN and SVR models by using multiple evaluation indicators. The multi-stand control strategy of the UVC work roll bending forces and shifting stroke is obtained according to the shape mechanism model and the prediction model based on the MOGPR method. Compared with the conventional control strategy, it can make the most of shifting stroke and the work roll bending forces fluctuates less, of which, F1, F2, and F3 are reduced by 21.85 %, 36.15 %, and 19.67 %, respectively. The industrial application of the control strategy during rolling achieves remarkable results that the measured high precision crown hit rate of electrical steel is significantly improved in the 1450 mm HSMs.

Expedient validation of LED reliability with anomaly detection through multi-output Gaussian process regression

The existing LED reliability assessment procedures are costly and time-consuming. It poses even greater challenges for the purchasers to validate the manufacturer test results, prior to goods acceptance. In absence of a simple and expedient validation, purchasers often bear the reliability risk in good faith. Leveraging on the 3-stage early degradation modes, as a signature, for LED under the moisture-electrical-temperature (MET) test, an expedient assessment framework is proposed to validate if the supplied LED batches conform to the contracted reliability. Multi-output Gaussian process regression (MOGPR) is performed on sparse reference data to establish an acceptable threshold for the degradation signature. When regressed against the reference data with contracted reliability, the test LED that displays signature anomaly will be deliberated for rejection. The acceptable threshold is calibrated to minimise false negative rate and false positive rate. The procedure is repeated in accordance with acceptable quality level (AQL) or existing sampling policies to accept or reject the batch. The framework achieves a reduction of the validation time from 6000 h to 100 h. While the initial motivation is to facilitate acceptance by purchasers, manufacturer may also adopt this approach for out-going quality control (QC) or rapid evaluation during design prototyping.

State of Health Trajectory Prediction Based on Multi-Output Gaussian Process Regression for Lithium-Ion Battery

Lithium-ion battery state of health (SOH) accurate prediction is of great significance to ensure the safe reliable operation of electric vehicles and energy storage systems. However, safety issues arising from the inaccurate estimation and prediction of battery SOH have caused widespread concern in academic and industrial communities. In this paper, a method is proposed to build an accurate SOH prediction model for battery packs based on multi-output Gaussian process regression (MOGPR) by employing the initial cycle data of the battery pack and the entire life cycling data of battery cells. Firstly, a battery aging experimental platform is constructed to collect battery aging data, and health indicators (HIs) that characterize battery aging are extracted. Then, the correlation between the HIs and the battery capacity is evaluated by the Pearson correlation analysis method, and the HIs that own a strong correlation to the battery capacity are screened. Finally, two MOGPR models are constructed to predict the HIs and SOH of the battery pack. Based on the first MOGPR model and the early HIs of the battery pack, the future cycle HIs can be predicted. In addition, the predicted HIs and the second MOGPR model are used to predict the SOH of the battery pack. The experimental results verify that the approach has a competitive performance; the mean and maximum values of the mean absolute error (MAE) and root mean square error (RMSE) are 1.07% and 1.42%, and 1.77% and 2.45%, respectively.

Long-term prediction enhancement based on multi-output Gaussian process regression integrated with production plans for oxygen supply network

The variation tendency of oxygen demand plays a crucial role in planning and scheduling in the steel industry. Traditional prediction methods merely utilize historical data without considering any future production information. In steel enterprises, the production plan is a kind of easy-to-obtain future production information, which specifies the load adjustment, start-up, and shutdown of equipment in the future. In this paper, a prediction enhancement method based on multi-output Gaussian process regression (MOGPR) for the oxygen demand prediction is first developed. The proposed method models the correlations over three data sources, i.e., the plan-based oxygen prediction, the history-based oxygen prediction, and the oxygen measurement, and transfers the meaningful knowledge between them. Two long-term oxygen demand prediction methods integrated with prediction enhancement are further proposed. Finally, two industrial case studies are used to compare and demonstrate the effectiveness and performance of the proposed methods.