Constrained Gaussian Process Research Articles

Bayesian quadrature policy optimization for spacecraft proximity maneuvers and docking

Advancing autonomous spacecraft proximity maneuvers and docking (PMD) is crucial for enhancing the efficiency and safety of inter-satellite services. One primary challenge in PMD is the accurate a priori definition of the system model, often complicated by inherent uncertainties in the system modeling and observational data. To address this challenge, we propose a novel Lyapunov Bayesian actor-critic reinforcement learning algorithm that guarantees the stability of the control policy under uncertainty. The PMD task is formulated as a Markov decision process that involves the relative dynamic model, the docking cone, and the cost function. By applying Lyapunov theory, we reformulate temporal difference learning as a constrained Gaussian process regression, enabling the state-value function to act as a Lyapunov function. Additionally, the proposed Bayesian quadrature policy optimization method analytically computes policy gradients, effectively addressing stability constraints while accommodating informational uncertainties in the PMD task. Experimental validation on a spacecraft air-bearing testbed demonstrates the significant and promising performance of the proposed algorithm.

Solving Bayesian inverse problems with expensive likelihoods using constrained Gaussian processes and active learning

Solving inverse problems using Bayesian methods can become prohibitively expensive when likelihood evaluations involve complex and large scale numerical models. A common approach to circumvent this issue is to approximate the forward model or the likelihood function with a surrogate model. But also there, due to limited computational resources, only a few training points are available in many practically relevant cases. Thus, it can be advantageous to model the additional uncertainties of the surrogate in order to incorporate the epistemic uncertainty due to limited data. In this paper, we develop a novel approach to approximate the log likelihood by a constrained Gaussian process based on prior knowledge about its boundedness. This improves the accuracy of the surrogate approximation without increasing the number of training samples. Additionally, we introduce a formulation to integrate the epistemic uncertainty due to limited training points into the posterior density approximation. This is combined with a state of the art active learning strategy for selecting training points, which allows to approximate posterior densities in higher dimensions very efficiently. We demonstrate the fast convergence of our approach for a benchmark problem and infer a random field that is discretized by 30 parameters using only about 1000 model evaluations. In a practically relevant example, the parameters of a reduced lung model are calibrated based on flow observations over time and voltage measurements from a coupled electrical impedance tomography simulation.

Multi-Agent Collaborative Bayesian Optimization via Constrained Gaussian Processes

The increase in the computational power of edge devices has opened a new paradigm for collaborative analytics whereby agents borrow strength from each other to improve their learning capabilities. This work focuses on collaborative Bayesian optimization (BO), in which agents work together to efficiently optimize black-box functions without the need for sensitive data exchange. Our idea revolves around introducing a class of constrained Gaussian process surrogates, enabling agents to borrow informative designs from high-performing collaborators to enhance and expedite their optimization process. Our approach presents the first general-purpose collaborative BO framework that is compatible with any Gaussian process kernel and most of the known acquisition functions. Despite the simplicity of our approach, we demonstrate that it offers elegant theoretical guarantees and significantly outperforms state-of-the-art methods, especially when agents have heterogeneous black-box functions. Through various simulations and a real-life experiment in additive manufacturing, we showcase the advantageous properties of our approach and the benefits derived from collaboration.

Nonlinear response modelling of material systems using constrained Gaussian processes

AbstractThis article investigates the suitability of constrained Gaussian process regression in predicting nonlinear mechanical responses of material systems with notably reduced uncertainties. This study reinforces the conventional Gaussian processes with mechanics‐informed prior knowledge observed in various kinematic responses. Stiffening and softening responses of material systems mostly demonstrate at least one of the boundedness, monotonicity and convexity conditions with respect to some kinematic variables. These relationships or impositions in turn are encoded into a constrained Gaussian process for prediction, uncertainty quantification and extrapolation. Using numerous examples and comparative studies, this article evidently proves that the use of constrained Gaussian processes is data‐efficient, highly accurate, yields low uncertainties, recovers model overfitting and extrapolates very well compared to unconstrained or conventional Gaussian processes. Moreover, the usability of the proposed numerical method across various engineering modelling domains such as multiscale homogenisation, experimentation, structural optimisation, material constitutive modelling and structural idealisation is demonstrated.

Input and Output Manifold Constrained Gaussian Process Regression for Galvanometric Setup Calibration

Data-driven techniques are finding their way into the calibration procedure of galvanometric setups. However, they bypass the underlying physical or mathematical model completely. Recent work has shown that a simple assumption about an underlying truth can improve the predictions: laser beams leaving the device follow a straight line. In this article, we take that approach a step further. Both the inputs (the pairs of rotations of the two mirrors) and outputs (the straight lines) lie on a manifold. We can incorporate this prior knowledge in the model via constraints built in the formulation of a covariance function. We propose two constrained models: one in which a linear constraint on the direction vector is written as a differential equation, and one in which a quadratic constraint is imposed by a reparametrization of the line coordinates. We compare them with the data-driven and unconstrained model-based approaches. We show that enforcing constraints improves the quality of the predictions significantly and, thus, the accuracy of the calibration. We validate our findings against real world data by predicting points on validation planes, calculating line segment distances, considering the training times for the models, and assessing how much a predicted line resembles an actual straight line.

A Hybrid Data Driven-Physics Constrained Gaussian Process Regression Framework with Deep Kernel for Uncertainty Quantification

A Hybrid Data Driven-Physics Constrained Gaussian Process Regression Framework with Deep Kernel for Uncertainty Quantification

Mathematical and simulation methods for deriving extinction thresholds in spatial and stochastic models of interacting agents

Abstract In ecology, one of the most fundamental questions relates to the persistence of populations, or conversely to the probability of their extinction. Deriving extinction thresholds and characterizing other critical phenomena in spatial and stochastic models is highly challenging, with few mathematically rigorous results being available for discrete‐space models such as the contact process. For continuous‐space models of interacting agents, to our knowledge no analytical results are available concerning critical phenomena, even if continuous‐space models can arguably be considered to be more natural descriptions of many ecological systems than lattice‐based models. Here we present both mathematical and simulation‐based methods for deriving extinction thresholds and other critical phenomena in a broad class of agent‐based models called spatiotemporal point processes. The mathematical methods are based on a perturbation expansion around the so‐called mean‐field model, which is obtained at the limit of large‐scale interactions. The simulation methods are based on examining how the mean time to extinction scales with the domain size used in the simulation. By utilizing a constrained Gaussian process fitted to the simulated data, the critical parameter value can be identified by asking when the scaling between logarithms of the time to extinction and the domain size switches from sublinear to superlinear. As a case study, we derive the extinction threshold for the spatial and stochastic logistic model. The mathematical technique yields rigorous approximation of the extinction threshold at the limit of long‐ranged interactions. The asymptotic validity of the approximation is illustrated by comparing it to simulation experiments. In particular, we show that species persistence is facilitated by either short or long spatial scale of the competition kernel, whereas an intermediate scale makes the species vulnerable to extinction. Both the mathematical and simulation methods developed here are of very general nature, and thus we expect them to be valuable for predicting many kinds of critical phenomena in continuous‐space stochastic models of interacting agents, and thus to be of broad interest for research in theoretical ecology and evolutionary biology.

Efficient Bayesian shape-restricted function estimation with constrained Gaussian process priors

This article revisits the problem of Bayesian shape-restricted inference in the light of a recently developed approximate Gaussian process that admits an equivalent formulation of the shape constraints in terms of the basis coefficients. We propose a strategy to efficiently sample from the resulting constrained posterior by absorbing a smooth relaxation of the constraint in the likelihood and using circulant embedding techniques to sample from the unconstrained modified prior. We additionally pay careful attention to mitigate the computational complexity arising from updating hyperparameters within the covariance kernel of the Gaussian process. The developed algorithm is shown to be accurate and highly efficient in simulated and real data examples.

Constrained Gaussian Process Learning for Model Predictive Control

Many control tasks can be formulated as tracking problems of a known or unknown reference signal. examples are motion compensation in collaborative robotics, the synchronisation of oscillations for power systems or the reference tracking of recipes in chemical process operation. Both the tracking performance and the stability of the closed-loop system depend strongly on two factors: Firstly, they depend on whether the future reference signal required for tracking is known, and secondly, whether the system can track the reference at all. This paper shows how to use machine learning, i.e. Gaussian processes, to learn a reference from (noisy) data while guaranteeing trackability of the modified desired reference predictions within the framework of model predictive control. Guarantees are provided by adjusting the hyperparameters via a constrained optimisation. Two specific scenarios, i.e. asymptotically constant and periodic REFERENCES, are discussed.

Numerical Solutions of Hamilton-Jacobi Inequalities by Constrained Gaussian Process Regression

This paper proposes numerical solutions of Hamilton-Jacobi inequalities based on constrained Gaussian process regression. While Gaussian process regression is a tool to estimate an unknown function...

Constrained Gaussian process with application in tissue-engineering scaffold biodegradation

ABSTRACTIn many biomanufacturing areas, such as tissue-engineering scaffold fabrication, the biodegradation performance of products is a key to producing products with desirable properties. The prediction of biodegradation often encounters the challenge of how to incorporate expert knowledge. This article proposes a Constrained Gaussian Process (CGP) method for predictive modeling with application to scaffold biodegradation. It provides a unified framework of using appropriate constraints to accommodate various types of expert knowledge in predictive modeling, including censoring, monotonicity, and bounds requirements. Efficient Bayesian sampling procedures for prediction are also developed. The performance of the proposed method is demonstrated in a case study on a novel scaffold fabrication process. Compared with the unconstrained GP and artificial neural networks, the proposed method can provide more accurate and meaningful prediction. A simulation study is also conducted to further reveal the properties of the CGP.

Advanced surrogate model and sensitivity analysis methods for sodium fast reactor accident assessment

Within the framework of the generation IV Sodium Fast Reactors, the safety in case of severe accidents is assessed. From this statement, CEA has developed a new physical tool to model the accident initiated by the Total Instantaneous Blockage (TIB) of a sub-assembly. This TIB simulator depends on many uncertain input parameters. This paper aims at proposing a global methodology combining several advanced statistical techniques in order to perform a global sensitivity analysis of this TIB simulator. The objective is to identify the most influential uncertain inputs for the various TIB outputs involved in the safety analysis. The proposed statistical methodology combining several advanced statistical techniques enables to take into account the constraints on the TIB simulator outputs (positivity constraints) and to deal simultaneously with various outputs. To do this, a space-filling design is used and the corresponding TIB model simulations are performed. Based on this learning sample, an efficient constrained Gaussian process metamodel is fitted on each TIB model outputs. Then, using the metamodels, classical sensitivity analyses are made for each TIB output. Multivariate global sensitivity analyses based on aggregated indices are also performed, providing additional valuable information. Main conclusions on the influence of each uncertain input are derived.

A constrained matrix-variate Gaussian process for transposable data

Transposable data represents interactions among two sets of entities, and are typically represented as a matrix containing the known interaction values. Additional side information may consist of feature vectors specific to entities corresponding to the rows and/or columns of such a matrix. Further information may also be available in the form of interactions or hierarchies among entities along the same mode (axis). We propose a novel approach for modeling transposable data with missing interactions given additional side information. The interactions are modeled as noisy observations from a latent noise free matrix generated from a matrix-variate Gaussian process. The construction of row and column covariances using side information provides a flexible mechanism for specifying a-priori knowledge of the row and column correlations in the data. Further, the use of such a prior combined with the side information enables predictions for new rows and columns not observed in the training data. In this work, we combine the matrix-variate Gaussian process model with low rank constraints. The constrained Gaussian process approach is applied to the prediction of hidden associations between genes and diseases using a small set of observed associations as well as prior covariances induced by gene-gene interaction networks and disease ontologies. The proposed approach is also applied to recommender systems data which involves predicting the item ratings of users using known associations as well as prior covariances induced by social networks. We present experimental results that highlight the performance of constrained matrix-variate Gaussian process as compared to state of the art approaches in each domain.