Multi-output Gaussian Process Research Articles

Geometrical deviation modeling and monitoring of 3D surface based on multi-output Gaussian process

Geometrical deviation is an important factor in determining the quality of a three-dimensional (3D) Surface. For 3D surfaces with complex shapes, the high-definition measurement (HDM) technology can provide detailed information on surface topography, which inspired new challenges in characterizing, modeling, and monitoring geometrical deviations. This paper proposes a spherical multi-output Gaussian process (S-MOGP) method to model and monitor 3D surfaces. Firstly, the surface in the 3D coordinate system is mapped to the spherical 2D parameter domain. Secondly, a state equation based on the multi-output Gaussian process is established to model the 3D surface. Finally, statistics are calculated and control charts are presented to monitor the geometrical deviation. The results of simulations and a case study show that the proposed method can effectively model 3D surfaces and monitor the geometrical deviations.

Knowledge-based turbomachinery design system via a deep neural network and multi-output Gaussian process

The requirements of future aeroengines challenge turbomachinery designs to be quieter, greener, and more efficient; furthermore, they must be developed at considerably reduced cycles and costs. Traditional turbomachinery design systems involve the iterative design–evaluation–improvement process, leading to high computational costs and considerable reliance on expert experience. In this paper, a knowledge-based turbomachinery aerodynamic design system consisting of a design of experiment framework and a two-stage modeling is proposed. In the first phase of the two-stage modeling, a deep neural network (DNN)-enhanced multi-output Gaussian process model is constructed. This model determines the optimal loading distribution from reference designs that are optimized for the desired performance. In the second stage, a DNN model is developed to predict the turbine profile geometry for a given loading distribution. This formulation enables the automatic determination of the optimal turbine geometry without an iterative process. Furthermore, the knowledge in the design system grows with the increasing database. The results indicate that the L2 relative errors for predicting the optimal pressure coefficient distribution and turbine profile geometry are less than 0.25% and 1.7%, respectively. A design case of a low-pressure turbine is used to validate the entire design system. The results reveal that the turbine geometry generated by the proposed design system outperforms that of the benchmark case in terms of the design and off-design aerodynamic performance. The foregoing observations are finally confirmed through experiments.

Nonlinear Online Multioutput Gaussian Process for Multistream Data Informatics

The revolution of Industry 4.0 and Internet of Things facilitate large and cheap data acquisition from complex systems. However, the online processing and analytics of such multi-stream data still suffer difficulties. The key reason is that data from different sensors (multi-stream data) contain nonlinear within-and-cross sensor correlations, which poses challenges in the modeling and analysis of such data, especially for the online scenarios. To jointly model multi-stream data, multi-output Gaussian Process (MGP) has become popular and achieved great success. However, existing methods assume different outputs are linearly correlated, which significantly limits their applications. In this paper, we propose a nonlinear online MGP framework to simultaneously realize nonlinear correlation modeling and online processing of multi-stream data. Specifically, a tailored co-variance structure is developed based on the convolution process, which can capture nonlinear self-and-cross correlations. To realize online processing, Bayesian analysis and marginalized particle filter are applied to estimate function values and model parameters. Comparing to existing methods, the proposed method features superiorities in i) the non-linear characterization of self-and-cross correlations, and ii) the efficient modeling and prediction capability. These features make the proposed method especially appropriate for industrial informatics with complex relationship and large-scale data sets. The proposed method is validated with simulation studies and a real-case study of biophysical signals.

Long-term prediction enhancement based on multi-output Gaussian process regression integrated with production plans for oxygen supply network

The variation tendency of oxygen demand plays a crucial role in planning and scheduling in the steel industry. Traditional prediction methods merely utilize historical data without considering any future production information. In steel enterprises, the production plan is a kind of easy-to-obtain future production information, which specifies the load adjustment, start-up, and shutdown of equipment in the future. In this paper, a prediction enhancement method based on multi-output Gaussian process regression (MOGPR) for the oxygen demand prediction is first developed. The proposed method models the correlations over three data sources, i.e., the plan-based oxygen prediction, the history-based oxygen prediction, and the oxygen measurement, and transfers the meaningful knowledge between them. Two long-term oxygen demand prediction methods integrated with prediction enhancement are further proposed. Finally, two industrial case studies are used to compare and demonstrate the effectiveness and performance of the proposed methods.

A marine propeller design method based on two-fidelity data levels

A Simulation Based Design Optimization method for marine propellers design using a two-fidelity levels metamodel for global design space exploration and optimization is presented. Response surfaces are built using the co-Kriging approximation, i.e. a multi-output Gaussian process that combines large low-fidelity dataset with few, costly, high-fidelity data. The method is applied for the hydrodynamic shape optimization of the E779A propeller using, as fidelity levels, two different physical models for the propeller performances prediction, namely an inviscid, potential based Boundary Element Method (low-fidelity) and a viscous, finite volume RANS solver (high-fidelity). Results demonstrate the feasibility of multi-objective, constrained, design procedures, like those involving marine propellers, using these multi-fidelity response surfaces. At the same time, the need of good correlations between low- and high-fidelity data feeding the response surfaces is highlighted as a requisite for robust and reliable predictions using these approximated methods.

Generalized multi-output Gaussian process censored regression

When modelling censored observations (i.e. data in which the value of a measurement or observation is un-observable beyond a given threshold), a typical approach in current regression methods is to use a censored-Gaussian (i.e. Tobit) model to describe the conditional output distribution. In this paper, as in the case of missing data, we argue that exploiting correlations between multiple outputs can enable models to better address the bias introduced by censored data. To do so, we introduce a heteroscedastic multi-output Gaussian process model which combines the non-parametric flexibility of GPs with the ability to leverage information from correlated outputs under input-dependent noise conditions. To address the resulting inference intractability, we further devise a variational bound to the marginal log-likelihood suitable for stochastic optimization. We empirically evaluate our model against other generative models for censored data on both synthetic and real world tasks and further show how it can be generalized to deal with arbitrary likelihood functions. Results show how the added flexibility allows our model to better estimate the underlying non-censored (i.e. true) process under potentially complex censoring dynamics.

Lithium battery state-of-health estimation and remaining useful lifetime prediction based on non-parametric aging model and particle filter algorithm

State of health estimation (SOH) and remaining useful lifetime (RUL) prediction are significant health indicators for improving the safety and reliability of battery systems. Herein, a data-fusion method is developed to establish a non-parametric degradation model and a particle filter algorithm for forecasting battery health conditions. Firstly, a dynamic battery aging state-space system is developed, in which Gaussian process regression is applied to establish state equation using historical capacity series and current capacity as input and output variables, respectively. Meanwhile, multi-output Gaussian process regression maps the relationship between capacity degradation and battery health indicators to construct an observation equation. Second, two filter methods are unitized to obtain the smooth differential thermal voltammetry curves and the significant health indicators are extracted from partial differential thermal voltammetry curves. Third, the short-term SOH estimation and long-term RUL prediction are carried out using a particle filter algorithm. Moreover, two types of five batteries with various designed cases are conducted to verify and analyze the proposed method. The results show that the estimation errors of short-term SOH are within 4% and prediction errors of long-term RUL are around 7% (relative error/EOL, 12/159), which indicate the proposed method has an excellent capability for accurate and robust forecasting battery health conditions. • A non-parametric state-space representation for battery degradation is developed. • Battery health conditions are estimated and predicted in multi-timescale perspective. • The health indicators are extracted from partial DTV curves in different dimensions. • Two types of five batteries with various designed cases are discussed and analyzed.

Modeling Neonatal EEG Using Multi-Output Gaussian Processes

Neonatal seizures are sudden events in brain activity with detrimental effects in neurological functions usually related to epileptic fits. Though neonatal seizures can be identified from electroencephalography (EEG), this is a challenging endeavour since expert visual inspection of EEG recordings is time consuming and prone to errors due the data’s nonstationarity and low signal-to-noise ratio. Towards the greater aim of automatic clinical decision making and monitoring, we propose a multi-output Gaussian process (MOGP) framework for neonatal EEG modelling. In particular, our work builds on the multi-output spectral mixture (MOSM) covariance kernel and shows that MOSM outperforms other commonly-used covariance functions in the literature when it comes to data imputation and hyperparameter-based seizure detection. To the best of our knowledge, our work is the first attempt at modelling and classifying neonatal EEG using MOGPs. Our main contributions are: i) the development of an MOGP-based framework for neonatal EEG analysis; ii) the experimental validation of the MOSM covariance kernel on real-world neonatal EEG for data imputation; and iii) the design of features for EEG based on MOSM hyperparameters and their validation for seizure detection (classification) in a patient specific approach.

Bayesian optimization of peripheral intraneural stimulation protocols to evoke distal limb movements

Objective. Motor neuroprostheses require the identification of stimulation protocols that effectively produce desired movements. Manual search for these protocols can be very time-consuming and often leads to suboptimal solutions, as several stimulation parameters must be personalized for each subject for a variety of target motor functions. Here, we present an algorithm that efficiently tunes peripheral intraneural stimulation protocols to elicit functionally relevant distal limb movements. Approach. We developed the algorithm using Bayesian optimization (BO) with multi-output Gaussian Processes (GPs) and defined objective functions based on coordinated muscle recruitment. We applied the algorithm offline to data acquired in rats for walking control and in monkeys for hand grasping control and compared different GP models for these two systems. We then performed a preliminary online test in a monkey to experimentally validate the functionality of our method. Main results. Offline, optimal intraneural stimulation protocols for various target motor functions were rapidly identified in both experimental scenarios. Using the model that performed best, the algorithm converged to stimuli that evoked functionally consistent movements with an average number of actions equal to 20% of the search space size in both the rat and monkey animal models. Online, the algorithm quickly guided the observations to stimuli that elicited functional hand gestures, although more selective motor outputs could have been achieved by refining the objective function used. Significance. These results demonstrate that BO can reliably and efficiently automate the tuning of peripheral neurostimulation protocols, establishing a translational framework to configure peripheral motor neuroprostheses in clinical applications. The proposed method can also potentially be applied to optimize motor functions using other stimulation modalities.

In Situ Blind Calibration of Sensor Networks for Infrastructure Monitoring

The recent development of Internet-of-Things (IoT) technologies has enabled smaller and lower-cost sensor nodes, motivating the deployment of more flexible and scalable sensor networks for infrastructure monitoring applications. However, because these nodes tend to be affected by environmental conditions and aging, they are prone to long-term drift over years of operation; thus, they need to be recalibrated on a regular basis to ensure data accuracy. In this paper, we propose an in situ blind calibration algorithm for infrastructure monitoring sensor networks, which requires neither physical intervention nor the assumption that the sensors are measuring identical ground-truth signals. The algorithm uses a multioutput Gaussian process (MOGP) to model the spatial-temporal distribution of the measurand and drift, thus removing irrelevant short-term fluctuations and decomposing the drift from long-term trends. We evaluate the algorithm on a real-world dataset, and the results show that the proposed method can successfully calibrate a number of drifting sensors more than 20% greater than previous methods while achieving a higher drift estimation accuracy.

Multioutput Gaussian processes with functional data: A study on coastal flood hazard assessment

Surrogate models are often used to replace costly-to-evaluate complex coastal codes to achieve substantial computational savings. In many of those models, the hydrometeorological forcing conditions (inputs) or flood events (outputs) are conveniently parameterized by scalar representations, neglecting that the inputs are actually time series and that floods propagate spatially inland. Both facts are crucial in flood prediction for complex coastal systems. Our aim is to establish a surrogate model that accounts for time-varying inputs and provides information on spatially varying inland flooding. We introduce a multioutput Gaussian process model based on a separable kernel that correlates both functional inputs and spatial locations. Efficient implementations consider tensor-structured computations or sparse-variational approximations. In several experiments, we demonstrate the versatility of the model for both learning maps and inferring unobserved maps, numerically showing the convergence of predictions as the number of learning maps increases. We assess our framework in a coastal flood prediction application. Predictions are obtained with small error values within computation time highly compatible with short-term forecast requirements (on the order of minutes compared to the days required by hydrodynamic simulators). We conclude that our framework is a promising approach for forecast and early-warning systems.

A data-fusion framework for lithium battery health condition Estimation Based on differential thermal voltammetry

Battery state foretasting and health management are significant tasks for ensuring safety and stability of battery systems. Accurate state estimation can not only provide valuable parameters for energy management but also may prolong battery usage lifespan. Comprehensive theoretical analysis and practical application, differential thermal voltammetry analysis method has great potentials in actual operation. This paper proposes a closed-loop battery capacity estimation framework, Gaussian process regression and multi-output Gaussian process regression for constructing battery dynamic state-space function, to improve the accuracy and robustness of battery SOH estimation. Firstly, a time-series model of battery capacity degradation is established as the state equation using Gaussian process regression. Secondly, two strong correlation indicators are treated as observed parameters to construct an observation equation through multi-output Gaussian process regression, where the health indicators are extracted from the partial smoothed curves by two filter methods. Thirdly, particle filter algorithm is employed to correct the prior estimated capacity and suppress noise perturbations for achieving closed-loop control. Additionally, the performances of particle filter algorithm with different particle sizes are discussed and analyzed from accuracy and computational time aspects. Verification of three types of batteries indicates that the proposed method has an excellent capability for battery capacity estimation.

Gradient-Enhanced Multi-Output Gaussian Process Model for Simulation-Based Engineering Design

The multi-output Gaussian process model has shown a promising way to deal with multiple related outputs. It can capture some useful information across outputs so as to provide more accurate predictions than simply modeling these outputs separately. If incorporating gradient formation into the modeling construction, the accuracy of the model can be further improved. The main original contribution of this work is to propose a multi-output Gaussian process model assisted by gradient information, which can enhance the prediction accuracies of multiple outputs simultaneously. The observed response values, as well as the gradient information of all the outputs, are incorporated in the covariance matrix. In such a structure, not only the observed responses but also the correlation information across different outputs can be fully used. The proposed model is demonstrated with two analytical examples and used for modeling aerodynamic coefficients of a NACA 0012 airfoil. Three other existing Gaussian-process-based models are also tested to compare with the proposed model. Results show that the proposed model is promising when it comes to the problems with multiple related outputs and the prediction accuracy can be enhanced with the help of gradient information.

Dynamic System Identification of Underwater Vehicles Using Multi-Output Gaussian Processes

Non-parametric system identification with Gaussian Processes for underwater vehicles is explored in this research with the purpose of modelling autonomous underwater vehicle (AUV) dynamics with low amount of data. Multi-output Gaussian processes and its aptitude to model the dynamic system of an underactuated AUV without losing the relationships between tied outputs is used. The simulation of a first-principles model of a Remus 100 AUV is employed to capture data for the training and validation of the multi-output Gaussian processes. The metric and required procedure to carry out multi-output Gaussian processes for AUV with 6 degrees of freedom (DoF) is also shown in this paper. Multi-output Gaussian processes are compared with the popular technique of recurrent neural network show that Multi-output Gaussian processes manage to surpass RNN for non-parametric dynamic system identification in underwater vehicles with highly coupled DoF with the added benefit of providing a measurement of confidence.

Optimization design of centrifugal pump impeller based on multi-output Gaussian process regression

To overcome the problems of large calculation cost and high dependence on designers’ experience, an optimization design method based on multi-output Gaussian process regression (MOGPR) was proposed. The hydraulic design method of centrifugal pump based on the MOGPR model was constructed under Bayesian framework. Based on the available excellent hydraulic model, the complex relationship between the performance parameters such as head, flow rate and the geometric parameters of centrifugal pump impeller was trained. The hydraulic design of the impeller for M125-100 centrifugal pump was performed by the proposed MOGPR surrogate model design method. The initial MOGPR design was further optimized by using the proposed MOGPR and NSGA-II hybrid model. The initial sample set for NSGA-II was designed by Latin hypercube design based on the MOGPR initial design. The relationship between the impeller geometry and the CFD numerical results of the sample set was trained to construct the surrogate model for pump hydraulic performance prediction. The MOGPR surrogate model was used to evaluate the objective function value of the offspring samples in NSGA-II multi-objective optimization. The comparison of the pump hydraulic performance between the optimized designs and the initial design shows that the efficiency and the head of the tradeoff optimal design are increased by 2.5% and 2.6%, respectively. The efficiency of the optimal head constraint design is increased by 3.2%. The comparison of the inner flow field shows that turbulent kinetic energy decreases significantly and flow separation is effectively suppressed for the optimal head constraint design.

Multi-output Gaussian process prediction for computationally expensive problems with multiple levels of fidelity

The multi-output Gaussian process (MOGP) modeling approach is a promising way to deal with multiple correlated outputs since it can capture useful information across outputs to provide more accurate predictions than simply modeling these outputs separately. While the existing MOGP models are considered as single-fidelity models and need a large number of high-fidelity samples to construct a relatively accurate surrogate model. The computational cost is often prohibitive, especially for computationally expensive problems. In this paper, a multi-fidelity multi-output Gaussian process (MMGP) model is proposed to deal with multi-fidelity (MF) data with multiple correlated outputs. It aims to approximate multiple correlated high-fidelity (HF) functions enhanced by the low-fidelity (LF) data. The MMGP model can make a trade-off between high prediction accuracy and low computational cost to relieve the computational burden. The performance of the MMGP model is demonstrated with two analytical examples with different data structures and an aerodynamic prediction example for a NACA 0012 airfoil. Another four well-known surrogate modeling methods are also tested to compare with the proposed approach. Results show that the proposed model is capable of dealing with diverse data structures, heterotopic data, or HF samples generated in different regions. The proposed model provides more accurate predictions than the other approaches for most test cases, especially when using the heterotopic training data. In the NACA0012 airfoil case, the root-mean-square error (RMSE) for the drag coefficient can be reduced by 30%–40% using the MMGP model compared with that of the other MF modeling methods.

Multi-objective aerodynamic optimization using active multi-output Gaussian process and mesh deformation method

A wing is an important part of the aircraft to improve aerodynamic performance. The current study is focused on an adaptive surrogate algorithm for airfoil aerodynamic optimization, which is based on a multi-output Gaussian process model. The conventional design method seriously relies on wind tunnel experiments and expensive computational simulations. The metamodels can significantly improve design efficiency and hence reduce the overall design costs. An active learning algorithm is proposed to improve the effectiveness of the multi-output Gaussian process model. The NSGA-II algorithm is adopted to obtain the optimal Pareto set with the optimization objectives of lift and drag coefficients for adaptive airfoil shapes. Besides, the Bezier curve and radial basis function are utilized in this study for airfoil mesh deformation. The results show that the airfoil shape can be obtained effectively by integrating the metamodel, active learning algorithm, and multi-objective optimization algorithm. The optimized results are of great engineering applications.

Multi-output Gaussian processes for multi-population longevity modelling

Abstract We investigate joint modelling of longevity trends using the spatial statistical framework of Gaussian process (GP) regression. Our analysis is motivated by the Human Mortality Database (HMD) that provides unified raw mortality tables for nearly 40 countries. Yet few stochastic models exist for handling more than two populations at a time. To bridge this gap, we leverage a spatial covariance framework from machine learning that treats populations as distinct levels of a factor covariate, explicitly capturing the cross-population dependence. The proposed multi-output GP models straightforwardly scale up to a dozen populations and moreover intrinsically generate coherent joint longevity scenarios. In our numerous case studies, we investigate predictive gains from aggregating mortality experience across nations and genders, including by borrowing the most recently available “foreign” data. We show that in our approach, information fusion leads to more precise (and statistically more credible) forecasts. We implement our models in R, as well as a Bayesian version in Stan that provides further uncertainty quantification regarding the estimated mortality covariance structure. All examples utilise public HMD datasets.

Symplectic Gaussian process regression of maps in Hamiltonian systems.

We present an approach to construct structure-preserving emulators for Hamiltonian flow maps and Poincaré maps based directly on orbit data. Intended applications are in moderate-dimensional systems, in particular, long-term tracing of fast charged particles in accelerators and magnetic plasma confinement configurations. The method is based on multi-output Gaussian process (GP) regression on scattered training data. To obtain long-term stability, the symplectic property is enforced via the choice of the matrix-valued covariance function. Based on earlier work on spline interpolation, we observe derivatives of the generating function of a canonical transformation. A product kernel produces an accurate implicit method, whereas a sum kernel results in a fast explicit method from this approach. Both are related to symplectic Euler methods in terms of numerical integration but fulfill a complementary purpose. The developed methods are first tested on the pendulum and the Hénon-Heiles system and results compared to spectral regression of the flow map with orthogonal polynomials. Chaotic behavior is studied on the standard map. Finally, the application to magnetic field line tracing in a perturbed tokamak configuration is demonstrated. As an additional feature, in the limit of small mapping times, the Hamiltonian function can be identified with a part of the generating function and thereby learned from observed time-series data of the system's evolution. For implicit GP methods, we demonstrate regression performance comparable to spectral bases and artificial neural networks for symplectic flow maps, applicability to Poincaré maps, and correct representation of chaotic diffusion as well as a substantial increase in performance for learning the Hamiltonian function compared to existing approaches.

Profile monitoring based on transfer learning of multiple profiles with incomplete samples

Profile monitoring is an important tool for quality control. Most existing profile monitoring approaches focus on monitoring a single profile. In practice, multiple profiles also widely exist and these profiles contain rich correlation information that can benefit the monitoring of interested/target profile. In this article, we propose a transfer learning framework to extract profile-to-profile inter-relationship to improve the monitoring performance. In this framework, profiles are modeled as a multi-output Gaussian process (MGP), and a specially designed covariance structure is proposed to reduce the computational load in optimizing the MGP parameters. More importantly, the proposed framework contains features for dealing with incomplete samples in each profile, which facilitates the information sharing among profiles with different data collection costs/availability. The proposed method is validated and compared with various benchmarks in extensive numerical studies and a case study of monitoring ice machine temperature profiles. The results show the proposed method can successfully transfer knowledge from related profiles to benefit the monitoring performance in the target profile. The R code of this paper would be available as on-line supplementary materials.