Multi-output Gaussian Process Research Articles

Modeling Neonatal EEG Using Multi-Output Gaussian Processes

Neonatal seizures are sudden events in brain activity with detrimental effects in neurological functions usually related to epileptic fits. Though neonatal seizures can be identified from electroencephalography (EEG), this is a challenging endeavour since expert visual inspection of EEG recordings is time consuming and prone to errors due the data’s nonstationarity and low signal-to-noise ratio. Towards the greater aim of automatic clinical decision making and monitoring, we propose a multi-output Gaussian process (MOGP) framework for neonatal EEG modelling. In particular, our work builds on the multi-output spectral mixture (MOSM) covariance kernel and shows that MOSM outperforms other commonly-used covariance functions in the literature when it comes to data imputation and hyperparameter-based seizure detection. To the best of our knowledge, our work is the first attempt at modelling and classifying neonatal EEG using MOGPs. Our main contributions are: i) the development of an MOGP-based framework for neonatal EEG analysis; ii) the experimental validation of the MOSM covariance kernel on real-world neonatal EEG for data imputation; and iii) the design of features for EEG based on MOSM hyperparameters and their validation for seizure detection (classification) in a patient specific approach.

Bayesian optimization of peripheral intraneural stimulation protocols to evoke distal limb movements

Objective. Motor neuroprostheses require the identification of stimulation protocols that effectively produce desired movements. Manual search for these protocols can be very time-consuming and often leads to suboptimal solutions, as several stimulation parameters must be personalized for each subject for a variety of target motor functions. Here, we present an algorithm that efficiently tunes peripheral intraneural stimulation protocols to elicit functionally relevant distal limb movements. Approach. We developed the algorithm using Bayesian optimization (BO) with multi-output Gaussian Processes (GPs) and defined objective functions based on coordinated muscle recruitment. We applied the algorithm offline to data acquired in rats for walking control and in monkeys for hand grasping control and compared different GP models for these two systems. We then performed a preliminary online test in a monkey to experimentally validate the functionality of our method. Main results. Offline, optimal intraneural stimulation protocols for various target motor functions were rapidly identified in both experimental scenarios. Using the model that performed best, the algorithm converged to stimuli that evoked functionally consistent movements with an average number of actions equal to 20% of the search space size in both the rat and monkey animal models. Online, the algorithm quickly guided the observations to stimuli that elicited functional hand gestures, although more selective motor outputs could have been achieved by refining the objective function used. Significance. These results demonstrate that BO can reliably and efficiently automate the tuning of peripheral neurostimulation protocols, establishing a translational framework to configure peripheral motor neuroprostheses in clinical applications. The proposed method can also potentially be applied to optimize motor functions using other stimulation modalities.

In Situ Blind Calibration of Sensor Networks for Infrastructure Monitoring

The recent development of Internet-of-Things (IoT) technologies has enabled smaller and lower-cost sensor nodes, motivating the deployment of more flexible and scalable sensor networks for infrastructure monitoring applications. However, because these nodes tend to be affected by environmental conditions and aging, they are prone to long-term drift over years of operation; thus, they need to be recalibrated on a regular basis to ensure data accuracy. In this paper, we propose an in situ blind calibration algorithm for infrastructure monitoring sensor networks, which requires neither physical intervention nor the assumption that the sensors are measuring identical ground-truth signals. The algorithm uses a multioutput Gaussian process (MOGP) to model the spatial-temporal distribution of the measurand and drift, thus removing irrelevant short-term fluctuations and decomposing the drift from long-term trends. We evaluate the algorithm on a real-world dataset, and the results show that the proposed method can successfully calibrate a number of drifting sensors more than 20% greater than previous methods while achieving a higher drift estimation accuracy.

Multioutput Gaussian processes with functional data: A study on coastal flood hazard assessment

Surrogate models are often used to replace costly-to-evaluate complex coastal codes to achieve substantial computational savings. In many of those models, the hydrometeorological forcing conditions (inputs) or flood events (outputs) are conveniently parameterized by scalar representations, neglecting that the inputs are actually time series and that floods propagate spatially inland. Both facts are crucial in flood prediction for complex coastal systems. Our aim is to establish a surrogate model that accounts for time-varying inputs and provides information on spatially varying inland flooding. We introduce a multioutput Gaussian process model based on a separable kernel that correlates both functional inputs and spatial locations. Efficient implementations consider tensor-structured computations or sparse-variational approximations. In several experiments, we demonstrate the versatility of the model for both learning maps and inferring unobserved maps, numerically showing the convergence of predictions as the number of learning maps increases. We assess our framework in a coastal flood prediction application. Predictions are obtained with small error values within computation time highly compatible with short-term forecast requirements (on the order of minutes compared to the days required by hydrodynamic simulators). We conclude that our framework is a promising approach for forecast and early-warning systems.

A data-fusion framework for lithium battery health condition Estimation Based on differential thermal voltammetry

Battery state foretasting and health management are significant tasks for ensuring safety and stability of battery systems. Accurate state estimation can not only provide valuable parameters for energy management but also may prolong battery usage lifespan. Comprehensive theoretical analysis and practical application, differential thermal voltammetry analysis method has great potentials in actual operation. This paper proposes a closed-loop battery capacity estimation framework, Gaussian process regression and multi-output Gaussian process regression for constructing battery dynamic state-space function, to improve the accuracy and robustness of battery SOH estimation. Firstly, a time-series model of battery capacity degradation is established as the state equation using Gaussian process regression. Secondly, two strong correlation indicators are treated as observed parameters to construct an observation equation through multi-output Gaussian process regression, where the health indicators are extracted from the partial smoothed curves by two filter methods. Thirdly, particle filter algorithm is employed to correct the prior estimated capacity and suppress noise perturbations for achieving closed-loop control. Additionally, the performances of particle filter algorithm with different particle sizes are discussed and analyzed from accuracy and computational time aspects. Verification of three types of batteries indicates that the proposed method has an excellent capability for battery capacity estimation.

Gradient-Enhanced Multi-Output Gaussian Process Model for Simulation-Based Engineering Design

The multi-output Gaussian process model has shown a promising way to deal with multiple related outputs. It can capture some useful information across outputs so as to provide more accurate predictions than simply modeling these outputs separately. If incorporating gradient formation into the modeling construction, the accuracy of the model can be further improved. The main original contribution of this work is to propose a multi-output Gaussian process model assisted by gradient information, which can enhance the prediction accuracies of multiple outputs simultaneously. The observed response values, as well as the gradient information of all the outputs, are incorporated in the covariance matrix. In such a structure, not only the observed responses but also the correlation information across different outputs can be fully used. The proposed model is demonstrated with two analytical examples and used for modeling aerodynamic coefficients of a NACA 0012 airfoil. Three other existing Gaussian-process-based models are also tested to compare with the proposed model. Results show that the proposed model is promising when it comes to the problems with multiple related outputs and the prediction accuracy can be enhanced with the help of gradient information.

Dynamic System Identification of Underwater Vehicles Using Multi-Output Gaussian Processes

Non-parametric system identification with Gaussian Processes for underwater vehicles is explored in this research with the purpose of modelling autonomous underwater vehicle (AUV) dynamics with low amount of data. Multi-output Gaussian processes and its aptitude to model the dynamic system of an underactuated AUV without losing the relationships between tied outputs is used. The simulation of a first-principles model of a Remus 100 AUV is employed to capture data for the training and validation of the multi-output Gaussian processes. The metric and required procedure to carry out multi-output Gaussian processes for AUV with 6 degrees of freedom (DoF) is also shown in this paper. Multi-output Gaussian processes are compared with the popular technique of recurrent neural network show that Multi-output Gaussian processes manage to surpass RNN for non-parametric dynamic system identification in underwater vehicles with highly coupled DoF with the added benefit of providing a measurement of confidence.

Optimization design of centrifugal pump impeller based on multi-output Gaussian process regression

To overcome the problems of large calculation cost and high dependence on designers’ experience, an optimization design method based on multi-output Gaussian process regression (MOGPR) was proposed. The hydraulic design method of centrifugal pump based on the MOGPR model was constructed under Bayesian framework. Based on the available excellent hydraulic model, the complex relationship between the performance parameters such as head, flow rate and the geometric parameters of centrifugal pump impeller was trained. The hydraulic design of the impeller for M125-100 centrifugal pump was performed by the proposed MOGPR surrogate model design method. The initial MOGPR design was further optimized by using the proposed MOGPR and NSGA-II hybrid model. The initial sample set for NSGA-II was designed by Latin hypercube design based on the MOGPR initial design. The relationship between the impeller geometry and the CFD numerical results of the sample set was trained to construct the surrogate model for pump hydraulic performance prediction. The MOGPR surrogate model was used to evaluate the objective function value of the offspring samples in NSGA-II multi-objective optimization. The comparison of the pump hydraulic performance between the optimized designs and the initial design shows that the efficiency and the head of the tradeoff optimal design are increased by 2.5% and 2.6%, respectively. The efficiency of the optimal head constraint design is increased by 3.2%. The comparison of the inner flow field shows that turbulent kinetic energy decreases significantly and flow separation is effectively suppressed for the optimal head constraint design.

Multi-output Gaussian process prediction for computationally expensive problems with multiple levels of fidelity

The multi-output Gaussian process (MOGP) modeling approach is a promising way to deal with multiple correlated outputs since it can capture useful information across outputs to provide more accurate predictions than simply modeling these outputs separately. While the existing MOGP models are considered as single-fidelity models and need a large number of high-fidelity samples to construct a relatively accurate surrogate model. The computational cost is often prohibitive, especially for computationally expensive problems. In this paper, a multi-fidelity multi-output Gaussian process (MMGP) model is proposed to deal with multi-fidelity (MF) data with multiple correlated outputs. It aims to approximate multiple correlated high-fidelity (HF) functions enhanced by the low-fidelity (LF) data. The MMGP model can make a trade-off between high prediction accuracy and low computational cost to relieve the computational burden. The performance of the MMGP model is demonstrated with two analytical examples with different data structures and an aerodynamic prediction example for a NACA 0012 airfoil. Another four well-known surrogate modeling methods are also tested to compare with the proposed approach. Results show that the proposed model is capable of dealing with diverse data structures, heterotopic data, or HF samples generated in different regions. The proposed model provides more accurate predictions than the other approaches for most test cases, especially when using the heterotopic training data. In the NACA0012 airfoil case, the root-mean-square error (RMSE) for the drag coefficient can be reduced by 30%–40% using the MMGP model compared with that of the other MF modeling methods.

Multi-objective aerodynamic optimization using active multi-output Gaussian process and mesh deformation method

A wing is an important part of the aircraft to improve aerodynamic performance. The current study is focused on an adaptive surrogate algorithm for airfoil aerodynamic optimization, which is based on a multi-output Gaussian process model. The conventional design method seriously relies on wind tunnel experiments and expensive computational simulations. The metamodels can significantly improve design efficiency and hence reduce the overall design costs. An active learning algorithm is proposed to improve the effectiveness of the multi-output Gaussian process model. The NSGA-II algorithm is adopted to obtain the optimal Pareto set with the optimization objectives of lift and drag coefficients for adaptive airfoil shapes. Besides, the Bezier curve and radial basis function are utilized in this study for airfoil mesh deformation. The results show that the airfoil shape can be obtained effectively by integrating the metamodel, active learning algorithm, and multi-objective optimization algorithm. The optimized results are of great engineering applications.

Multi-output Gaussian processes for multi-population longevity modelling

Abstract We investigate joint modelling of longevity trends using the spatial statistical framework of Gaussian process (GP) regression. Our analysis is motivated by the Human Mortality Database (HMD) that provides unified raw mortality tables for nearly 40 countries. Yet few stochastic models exist for handling more than two populations at a time. To bridge this gap, we leverage a spatial covariance framework from machine learning that treats populations as distinct levels of a factor covariate, explicitly capturing the cross-population dependence. The proposed multi-output GP models straightforwardly scale up to a dozen populations and moreover intrinsically generate coherent joint longevity scenarios. In our numerous case studies, we investigate predictive gains from aggregating mortality experience across nations and genders, including by borrowing the most recently available “foreign” data. We show that in our approach, information fusion leads to more precise (and statistically more credible) forecasts. We implement our models in R, as well as a Bayesian version in Stan that provides further uncertainty quantification regarding the estimated mortality covariance structure. All examples utilise public HMD datasets.

Symplectic Gaussian process regression of maps in Hamiltonian systems.

We present an approach to construct structure-preserving emulators for Hamiltonian flow maps and Poincaré maps based directly on orbit data. Intended applications are in moderate-dimensional systems, in particular, long-term tracing of fast charged particles in accelerators and magnetic plasma confinement configurations. The method is based on multi-output Gaussian process (GP) regression on scattered training data. To obtain long-term stability, the symplectic property is enforced via the choice of the matrix-valued covariance function. Based on earlier work on spline interpolation, we observe derivatives of the generating function of a canonical transformation. A product kernel produces an accurate implicit method, whereas a sum kernel results in a fast explicit method from this approach. Both are related to symplectic Euler methods in terms of numerical integration but fulfill a complementary purpose. The developed methods are first tested on the pendulum and the Hénon-Heiles system and results compared to spectral regression of the flow map with orthogonal polynomials. Chaotic behavior is studied on the standard map. Finally, the application to magnetic field line tracing in a perturbed tokamak configuration is demonstrated. As an additional feature, in the limit of small mapping times, the Hamiltonian function can be identified with a part of the generating function and thereby learned from observed time-series data of the system's evolution. For implicit GP methods, we demonstrate regression performance comparable to spectral bases and artificial neural networks for symplectic flow maps, applicability to Poincaré maps, and correct representation of chaotic diffusion as well as a substantial increase in performance for learning the Hamiltonian function compared to existing approaches.

Profile monitoring based on transfer learning of multiple profiles with incomplete samples

Profile monitoring is an important tool for quality control. Most existing profile monitoring approaches focus on monitoring a single profile. In practice, multiple profiles also widely exist and these profiles contain rich correlation information that can benefit the monitoring of interested/target profile. In this article, we propose a transfer learning framework to extract profile-to-profile inter-relationship to improve the monitoring performance. In this framework, profiles are modeled as a multi-output Gaussian process (MGP), and a specially designed covariance structure is proposed to reduce the computational load in optimizing the MGP parameters. More importantly, the proposed framework contains features for dealing with incomplete samples in each profile, which facilitates the information sharing among profiles with different data collection costs/availability. The proposed method is validated and compared with various benchmarks in extensive numerical studies and a case study of monitoring ice machine temperature profiles. The results show the proposed method can successfully transfer knowledge from related profiles to benefit the monitoring performance in the target profile. The R code of this paper would be available as on-line supplementary materials.

Hidden physics model for parameter estimation of elastic wave equations

A numerical approach based on the hidden physics model to estimate the model parameters of elastic wave equations with the sparse and noisy data is presented in this paper. Through discretizing the time derivatives of elastic wave equations and placing the priors of the state variables as Gaussian process, the model parameters and structure of elastic wave equations are encoded in the kernel function of a multi-output Gaussian process. In the learning stage, a parameter bound constraint condition is incorporated to enforce the physical bound of the model parameters. The numerical results from several benchmark problems, including homogeneous media, layer media, anisotropic media, and homogeneous model with an inclusion, demonstrate the feasibility and performance of the hidden physics model.

Efficacy of Data Fusion Using Convolved Multi-Output Gaussian Processes

This paper evaluates the efficacy of a machine learning approach to data fusion using convolved multi-output Gaussian processes in the context of geological resource modeling. It empirically demonstrates that information integration across multiple information sources leads to superior estimates of all the quantities being modeled, compared to modeling them individually. Convolved multi-output Gaussian processes provide a powerful approach for simultaneous modeling of multiple quantities of interest while taking correlations between these quantities into consideration. Experiments are performed on large scale data taken from a mining context.

Neyman type A Distribution for the Natural Disasters and Related Casualties in Turkey

This paper evaluates the efficacy of a machine learning approach to data fusion using convolved multi-output Gaussian processes in the context of geological resource modeling. It empirically demonstrates that information integration across multiple information sources leads to superior estimates of all the quantities being modeled, compared to modeling them individually. Convolved multi-output Gaussian processes provide a powerful approach for simultaneous modeling of multiple quantities of interest while taking correlations between these quantities into consideration. Experiments are performed on large scale data taken from a mining context.

UAV flight control sensing enhancement with a data-driven adaptive fusion model

Accurate sensing is essential for achieving reliable control of unmanned aerial vehicles (UAVs). In prior works, the unscented Kalman filter (UKF) has shown superior performance in both estimation accuracy and computation efficiency, which makes it suitable for onboard sensing enhancement. However, the prediction accuracy of existing filter-based statistical models is generally assumed to be invariant of the flight conditions. To avoid deterioration in the estimation performance, this paper proposes a novel data-driven adaptive fusion state space model for quantifying the prediction uncertainty of the system model. Based on the multi-output Gaussian process regression (GPR), a rule for tuning the noise parameter of the statistical model is provided based on the estimated variance. The sparse GPR model is utilized to incorporate available features and obtain high estimation accuracy under dynamic operation conditions. Simulation results have illustrated the superior performance of the proposed approach in state estimation for the UAV.

Integrating Domain Knowledge in Data-Driven Earth Observation With Process Convolutions

The modelling of Earth observation data is a challenging problem, typically approached by either purely mechanistic or purely data-driven methods. Mechanistic models encode the domain knowledge and physical rules governing the system. Such models, however, need the correct specification of all interactions between variables in the problem and the appropriate parameterization is a challenge in itself. On the other hand, machine learning approaches are flexible data-driven tools, able to approximate arbitrarily complex functions, but lack interpretability and struggle when data is scarce or in extrapolation regimes. In this paper, we argue that hybrid learning schemes that combine both approaches can address all these issues efficiently. We introduce Gaussian process (GP) convolution models for hybrid modelling in Earth observation (EO) problems. We specifically propose the use of a class of GP convolution models called latent force models (LFMs) for EO time series modelling, analysis and understanding. LFMs are hybrid models that incorporate physical knowledge encoded in differential equations into a multioutput GP model. LFMs can transfer information across time-series, cope with missing observations, infer explicit latent functions forcing the system, and learn parameterizations which are very helpful for system analysis and interpretability. We consider time series of soil moisture from active (ASCAT) and passive (SMOS, AMSR2) microwave satellites. We show how assuming a first order differential equation as governing equation, the model automatically estimates the e-folding time or decay rate related to soil moisture persistence and discovers latent forces related to precipitation. The proposed hybrid methodology reconciles the two main approaches in remote sensing parameter estimation by blending statistical learning and mechanistic modeling.

Machine learning assisted stochastic-XFEM for stochastic crack propagation and reliability analysis

We present a novel stochastic extended finite element (S-XFEM) method for solving fracture mechanics problems under uncertainty. The proposed S-XFEM couples XFEM and a novel multi-output Gaussian process based machine learning algorithm, referred to as the hybrid polynomial correlated function expansion (H-PCFE). With this approach, the underlying stochastic fracture mechanics problem is decoupled into multiple classical fracture mechanics problems. Since solution of a classical fracture mechanics problem is computationally expensive, we utilize the H-PCFE model as a surrogate to emulate the behavior of the system. Training samples required for training the H-PCFE model are generated by using the XFEM. Because of the intrinsic capability of XFEM in providing a mesh insensitive solution, the proposed approach can provide reasonable result from a coarse mesh. On the other hand, H-PCFE is capable of providing highly accurate solution from very few training samples. Overall, the proposed S-XFEM is highly efficient in solving stochastic fracture mechanics problems. The proposed approach is used for solving three stochastic fracture mechanics problems. Different case studies involving reliability analysis and stochastic fracture propagation have been reported. The procedure yields highly accurate results for all problems, indicating its possible applications to other large scale systems.

Bias Suppression Framework for Detrending Mean of Multi-Output Gaussian Process Regression in LED Remaining Storage Life Prognosis

Bias Suppression Framework for Detrending Mean of Multi-Output Gaussian Process Regression in LED Remaining Storage Life Prognosis