Use Of Gaussian Processes Research Articles

Stochastic Modeling of Distance to Collision for Robot Manipulators

Evaluating distance to collision for robot manipulators is useful for assessing the feasibility of a robot configuration or for defining safe robot motion in unpredictable environments. However, distance estimation is a time-consuming operation, and the sensors involved in measuring the distance are always noisy. A challenge thus exists in evaluating the expected distance to collision for safer robot control and planning. In this work, we propose the use of Gaussian process (GP) regression and the forward kinematics (FK) kernel (a similarity function for robot manipulators) to efficiently and accurately estimate distance to collision. We show that the GP model with the FK kernel achieves almost 70 times faster distance evaluations compared to a standard geometric technique, and up to 18 times more accurate evaluations compared to other regression models, even when the GP is trained on noisy distance measurements. We employ this technique in trajectory optimization tasks and observe 13 times faster optimization than with the noise-free geometric approach yet obtain similar optimized motion plans. We also propose a confidence-based hybrid model that uses model-based predictions in regions of high confidence and switches to a more expensive sensor-based approach in other areas, and we demonstrate the usefulness of this hybrid model in an application involving reaching into a narrow passage.

A Methodology for the Statistical Calibration of Complex Constitutive Material Models: Application to Temperature-Dependent Elasto-Visco-Plastic Materials.

The calibration of any sophisticated model, and in particular a constitutive relation, is a complex problem that has a direct impact in the cost of generating experimental data and the accuracy of its prediction capacity. In this work, we address this common situation using a two-stage procedure. In order to evaluate the sensitivity of the model to its parameters, the first step in our approach consists of formulating a meta-model and employing it to identify the most relevant parameters. In the second step, a Bayesian calibration is performed on the most influential parameters of the model in order to obtain an optimal mean value and its associated uncertainty. We claim that this strategy is very efficient for a wide range of applications and can guide the design of experiments, thus reducing test campaigns and computational costs. Moreover, the use of Gaussian processes together with Bayesian calibration effectively combines the information coming from experiments and numerical simulations. The framework described is applied to the calibration of three widely employed material constitutive relations for metals under high strain rates and temperatures, namely, the Johnson–Cook, Zerilli–Armstrong, and Arrhenius models.

The role of surrogate models in the development of digital twins of dynamic systems

Digital twin technology has significant promise, relevance and potential of widespread applicability in various industrial sectors such as aerospace, infrastructure and automotive. However, the adoption of this technology has been slower due to the lack of clarity for specific applications. A discrete damped dynamic system is used in this paper to explore the concept of a digital twin. As digital twins are also expected to exploit data and computational methods, there is a compelling case for the use of surrogate models in this context. Motivated by this synergy, we have explored the possibility of using surrogate models within the digital twin technology. In particular, the use of Gaussian process (GP) emulator within the digital twin technology is explored. GP has the inherent capability of addressing noisy and sparse data and hence, makes a compelling case to be used within the digital twin framework. Cases involving stiffness variation and mass variation are considered, individually and jointly, along with different levels of noise and sparsity in data. Our numerical simulation results clearly demonstrate that surrogate models, such as GP emulators, have the potential to be an effective tool for the development of digital twins. Aspects related to data quality and sampling rate are analysed. Key concepts introduced in this paper are summarised and ideas for urgent future research needs are proposed.

Crop Yield Estimation and Interpretability With Gaussian Processes

This work introduces the use of Gaussian processes (GPs) for the estimation and understanding of crop development and yield using multisensor satellite observations and meteorological data. The proposed methodology combines synergistic information on canopy greenness, biomass, soil, and plant water content from optical and microwave sensors with the atmospheric variables typically measured at meteorological stations. A composite covariance is used in the GP model to account for varying scales, nonstationary, and nonlinear processes. The GP model reports noticeable gains in terms of accuracy with respect to other machine learning approaches for the estimation of corn, wheat, and soybean yields consistently for four years of data across continental U.S. (CONUS). Sparse GPs allow obtaining fast and compact solutions up to a limit, where heavy sparsity compromises the credibility of confidence intervals. We further study the GP interpretability by sensitivity analysis, which reveals that remote sensing parameters accounting for soil moisture and greenness mainly drive the model predictions. GPs finally allow us to identify climate extremes and anomalies impacting crop productivity and their associated drivers.

Gaussian process regression for seismic fragility assessment of building portfolios

Seismic fragility assessment of building portfolios is often based on the analysis of “average” building models representative of structural types (or building classes), thus neglecting building-to-building variability within a structural type. This paper proposes the use of Gaussian process (GP) regressions to develop flexible and accurate metamodels explicitly mapping building-class attributes to the seismic fragility parameters. The proposed metamodels can enable analysts to account for building-to-building variability in simulation-based seismic risk assessment of building portfolios. Unlike other commonly-used metamodels, GP regressions do not require the a-priori definition of a prediction function and they quantify the uncertainty on the predictions in a refined and explicit fashion. The proposed method is demonstrated for a portfolio of seismically-deficient reinforced concrete school buildings with construction details typical of some developing countries. Based on the available information about the building attributes (e.g. geometry, materials, detailing), building realisations are generated based on two alternative approaches, which are critically compared: design of experiment and Monte Carlo sampling. Cloud-based time-history analysis for each building realisation is performed using unscaled real ground-motion records; fragility relationships are derived for four structure-specific damage states. A GP regression is then developed for each considered fragility parameter (i.e. median and dispersion). To further increase the tractability and scalability of the methodology, alternative metamodels are defined based on numerical non-linear static pushover analyses or analytical “by-hand” pushover analyses, through the Simple Lateral Mechanism Analysis (SLaMA) method. The results show that, for the considered portfolio, the fitted GP regressions have a high predictive power in surrogating the modelled fragility, demonstrating the feasibility of the approach in practice. It is also shown that the choice of the sampling technique could be based on the input data availability, rather than on the expected computational burden. Finally, the use of simplified methods for response analysis shows acceptable error levels with respect to the full time-history analysis results. Such simplified methods can be promising alternatives to generate large training datasets for the proposed GP regressions. This increases the potential of training metamodels in practical portfolio risk assessment applications, in which a high number of building types, each characterised by a large number of attributes, is generally involved.

Multi-Objective Multidisciplinary Design Optimization Approach for Partially Reusable Launch Vehicle Design

Reusability of the first stage of launch vehicles may offer new perspectives to lower the cost of payload injection into orbit if sufficient reliability and efficient refurbishment can be achieved. One possible option that may be explored is to design the vehicle first stage for both reusable and expendable uses, in order to increase the flexibility and adaptability to different target missions. This paper proposes a multilevel multidisciplinary design optimization (MDO) approach to design aerospace vehicles addressing multimission problems. The proposed approach is focused on the design of a family of launchers for different missions sharing commonalities using multi-objective MDO to account for the computational cost associated with the discipline simulations. The multimission problem addressed considers two missions: 1) a reusable configuration for a sun synchronous orbit with a medium payload range and recovery of the first stage using a gliding-back strategy; 2) an expendable configuration for a medium payload injected into a geostationary transfer orbit. A dedicated MDO formulation introducing couplings between the missions is proposed in order to efficiently solve such a coupled problem while limiting the number of calls to the exact multidisciplinary analysis thanks to the use of Gaussian processes and multi-objective efficient global optimization.

Bayesian anomaly detection and classification for noisy data

Statistical uncertainties are rarely incorporated into machine learning algorithms, especially for anomaly detection. Here we present the Bayesian Anomaly Detection And Classification (BADAC) formalism, which provides a unified statistical approach to classification and anomaly detection within a hierarchical Bayesian framework. BADAC deals with uncertainties by marginalising over the unknown, true, value of the data. Using simulated data with Gaussian noise as an example, BADAC is shown to be superior to standard algorithms in both classification and anomaly detection performance in the presence of uncertainties. Additionally, BADAC provides well-calibrated classification probabilities, valuable for use in scientific pipelines. We show that BADAC can work in online mode and is fairly robust to model errors, which can be diagnosed through model-selection methods. In addition it can perform unsupervised new class detection and can naturally be extended to search for anomalous subsets of data. BADAC is therefore ideal where computational cost is not a limiting factor and statistical rigour is important. We discuss approximations to speed up BADAC, such as the use of Gaussian processes, and finally introduce a new metric, the Rank-Weighted Score (RWS), that is particularly suited to evaluating an algorithm’s ability to detect anomalies.

On learning Hamiltonian systems from data.

Concise, accurate descriptions of physical systems through their conserved quantities abound in the natural sciences. In data science, however, current research often focuses on regression problems, without routinely incorporating additional assumptions about the system that generated the data. Here, we propose to explore a particular type of underlying structure in the data: Hamiltonian systems, where an "energy" is conserved. Given a collection of observations of such a Hamiltonian system over time, we extract phase space coordinates and a Hamiltonian function of them that acts as the generator of the system dynamics. The approach employs an autoencoder neural network component to estimate the transformation from observations to the phase space of a Hamiltonian system. An additional neural network component is used to approximate the Hamiltonian function on this constructed space, and the two components are trained jointly. As an alternative approach, we also demonstrate the use of Gaussian processes for the estimation of such a Hamiltonian. After two illustrative examples, we extract an underlying phase space as well as the generating Hamiltonian from a collection of movies of a pendulum. The approach is fully data-driven and does not assume a particular form of the Hamiltonian function.

Replicated Computations Results (RCR) Report for “Statistical Abstraction for Multi-scale Spatio-temporal Systems”

“Statistical abstraction for multi-scale spatio-temporal systems” proposes a methodology that supports analysis of large-scaled spatio-temporal systems. These are represented via a set of agents whose behaviour depends on a perceived field. The proposed approach is based on a novel simulation strategy based on a statistical abstraction of the agents. The abstraction makes use of Gaussian Processes, a powerful class of non-parametric regression techniques from Bayesian Machine Learning, to estimate the agent’s behaviour given the environmental input. The authors use two biological case studies to show how the proposed technique can be used to speed up simulations and provide further insights into model behaviour. This replicated computation results report focuses on the scripts used in the paper to perform such analysis. The required software was straightforward to install and use. All the experimental results from the paper have been reproduced.

Statistical Abstraction for Multi-scale Spatio-temporal Systems

Modelling spatio-temporal systems exhibiting multi-scale behaviour is a powerful tool in many branches of science, yet it still presents significant challenges. Here, we consider a general two-layer (agent-environment) modelling framework, where spatially distributed agents behave according to external inputs and internal computation; this behaviour may include influencing their immediate environment, creating a medium over which agent-agent interaction signals can be transmitted. We propose a novel simulation strategy based on a statistical abstraction of the agent layer, which is typically the most detailed component of the model and can incur significant computational cost in simulation. The abstraction makes use of Gaussian Processes, a powerful class of non-parametric regression techniques from Bayesian Machine Learning, to estimate the agent’s behaviour given the environmental input. We show on two biological case studies how this technique can be used to speed up simulations and provide further insights into model behaviour.

Quantifying correlated truncation errors in effective field theory

Effective field theories (EFTs) organize the description of complex systems into an infinite sequence of decreasing importance. Predictions are made with a finite number of terms, which induces a truncation error that is often left unquantified. We formalize the notion of EFT convergence and propose a Bayesian truncation error model for predictions that are correlated across the independent variables, e.g., energy or scattering angle. Central to our approach are Gaussian processes that encode both the naturalness and correlation structure of EFT coefficients. Our use of Gaussian processes permits efficient and accurate assessment of credible intervals, allows EFT fits to easily include correlated theory errors, and provides analytic posteriors for physical EFT-related quantities such as the expansion parameter. We demonstrate that model-checking diagnostics---applied to the case of multiple curves---are powerful tools for EFT validation. As an example, we assess a set of nucleon-nucleon scattering observables in chiral EFT. In an effort to be self contained, appendices include thorough derivations of our statistical results. Our methods are packaged in Python code, called gsum, that is available for download on GitHub.

Deep Bayesian Optimization on Attributed Graphs

Attributed graphs, which contain rich contextual features beyond just network structure, are ubiquitous and have been observed to benefit various network analytics applications. Graph structure optimization, aiming to find the optimal graphs in terms of some specific measures, has become an effective computational tool in complex network analysis. However, traditional model-free methods suffer from the expensive computational cost of evaluating graphs; existing vectorial Bayesian optimization methods cannot be directly applied to attributed graphs and have the scalability issue due to the use of Gaussian processes (GPs). To bridge the gap, in this paper, we propose a novel scalable Deep Graph Bayesian Optimization (DGBO) method on attributed graphs. The proposed DGBO prevents the cubical complexity of the GPs by adopting a deep graph neural network to surrogate black-box functions, and can scale linearly with the number of observations. Intensive experiments are conducted on both artificial and real-world problems, including molecular discovery and urban road network design, and demonstrate the effectiveness of the DGBO compared with the state-of-the-art.

Exploring Bayesian parameter estimation for chiral effective field theory using nucleon–nucleon phase shifts

We recently developed a Bayesian framework for parameter estimation in general effective field theories. Here we present selected results from using that framework to estimate parameters with a nucleon–nucleon (NN) potential derived using chiral effective field theory (χEFT): the semi-local NN potential of Epelbaum, Krebs, and Meißner (EKM). There are many NN scattering data, up to high energies, and with rather small errors. Imposing a penalty for unnatural low-energy constants (LECs) therefore has a small effect on the fits, unless only a small fraction of the NN data is used in the fit. In contrast, we have found that including an estimate of higher orders in χEFT plays an important role in robust parameter estimation. We present two case studies where our Bayesian machinery illuminates physics issues. The first involves the EKM potential at fourth order in the χEFT expansion: the two-dimensional posterior probability density function (pdf) for the fourth-order s-wave LECs obtained from the Nijmegen PWA93 phase shifts indicates these parameters in the NN potential are degenerate. We trace this feature of the pdf to the presence of an operator in the fourth-order NN potential that vanishes on-shell. The second case study examines the stability of LEC extractions as more data at higher energies are included in the fit. We show that as long as χEFT truncation errors are properly accounted for in the parameter estimation, the LEC values extracted using our Bayesian approach are not sensitive to the maximum energy chosen for the fit. Uncorrelated and fully correlated models for the truncation errors are compared, pointing the way to the use of Gaussian processes to more generally model the correlation structure.

Gaussian process based optimization of molecular geometries using statistically sampled energy surfaces from quantum Monte Carlo.

Optimization of atomic coordinates and lattice parameters remains a significant challenge to the wide use of stochastic electronic structure methods such as quantum Monte Carlo (QMC). Measurements of the forces and stress tensor by these methods contain statistical errors, challenging conventional gradient-based numerical optimization methods that assume deterministic results. Additionally, forces are not yet available for some methods, wavefunctions, and basis sets and when available may be expensive to compute to sufficiently high statistical accuracy near energy minima, where the energy surfaces are flat. Here, we explore the use of Gaussian process based techniques to sample the energy surfaces and reduce sensitivity to the statistical nature of the problem. We utilize Latin hypercube sampling, with the number of sampled energy points scaling quadratically with the number of optimized parameters. We show these techniques may be successfully applied to systems consisting of tens of parameters, demonstrating QMC optimization of a benzene molecule starting from a randomly perturbed, broken symmetry geometry.

A data-driven photometric stereo algorithm

Representing the complicated reflectance of non-Lambertian material is one of the difficulty in photometric stereo. This paper presents a Gaussian process based data-driven photometric stereo method to address this problem. The method makes use of Gaussian process, a non-parametric Bayesian inference model, to establish a global plausible model to represent non-linear reflectance property of non-Lambertian material based on measured bidirectional reflectance distribution functions. The statistic nature of the Gaussian process makes the method has much better flexibility and accuracy in describing the complicated reflectance of various non-Lambterian materials so that a better estimation accuracy can be achieved for a photometric stereo system. The validity of the proposed method is verified by both computer simulation and real experiment. The results show that the proposed method has much better accuracy and efficiency in estimating surface normal than other existing methods.

Model selection using cosmic chronometers with Gaussian Processes

The use of Gaussian Processes with a measurement of the cosmic expansion rate based solely on the observation of cosmic chronometers provides a completely cosmology-independent reconstruction of the Hubble constant H(z) suitable for testing different models. The corresponding dispersion σH is smaller than ∼ 9% over the entire redshift range (≲ z≲ 20) of the observations, rivaling many kinds of cosmological measurements available today. We use the reconstructed H(z) function to test six different cosmologies, and show that it favours the Rh=ct universe, which has only one free parameter (i.e., H0) over other models, including Planck ΛCDM . The parameters of the standard model may be re-optimized to improve the fits to the reconstructed H(z) function, but the results have smaller p-values than one finds with Rh=ct.

Uncertainty-based simulation-optimization using Gaussian process emulation: Application to coastal groundwater management

Combined simulation-optimization (S/O) schemes have long been recognized as a valuable tool in coastal groundwater management (CGM). However, previous applications have mostly relied on deterministic seawater intrusion (SWI) simulations. This is a questionable simplification, knowing that SWI models are inevitably prone to epistemic and aleatory uncertainty, and hence a management strategy obtained through S/O without consideration of uncertainty may result in significantly different real-world outcomes than expected. However, two key issues have hindered the use of uncertainty-based S/O schemes in CGM, which are addressed in this paper. The first issue is how to solve the computational challenges resulting from the need to perform massive numbers of simulations. The second issue is how the management problem is formulated in presence of uncertainty. We propose the use of Gaussian process (GP) emulation as a valuable tool in solving the computational challenges of uncertainty-based S/O in CGM. We apply GP emulation to the case study of Kish Island (located in the Persian Gulf) using an uncertainty-based S/O algorithm which relies on continuous ant colony optimization and Monte Carlo simulation. In doing so, we show that GP emulation can provide an acceptable level of accuracy, with no bias and low statistical dispersion, while tremendously reducing the computational time. Moreover, five new formulations for uncertainty-based S/O are presented based on concepts such as energy distances, prediction intervals and probabilities of SWI occurrence. We analyze the proposed formulations with respect to their resulting optimized solutions, the sensitivity of the solutions to the intended reliability levels, and the variations resulting from repeated optimization runs.

Nonparametric estimation of stochastic differential equations with sparse Gaussian processes.

The application of stochastic differential equations (SDEs) to the analysis of temporal data has attracted increasing attention, due to their ability to describe complex dynamics with physically interpretable equations. In this paper, we introduce a nonparametric method for estimating the drift and diffusion terms of SDEs from a densely observed discrete time series. The use of Gaussian processes as priors permits working directly in a function-space view and thus the inference takes place directly in this space. To cope with the computational complexity that requires the use of Gaussian processes, a sparse Gaussian process approximation is provided. This approximation permits the efficient computation of predictions for the drift and diffusion terms by using a distribution over a small subset of pseudosamples. The proposed method has been validated using both simulated data and real data from economy and paleoclimatology. The application of the method to real data demonstrates its ability to capture the behavior of complex systems.

Characterizing Dynamic Walking Patterns and Detecting Falls with Wearable Sensors Using Gaussian Process Methods.

By incorporating a growing number of sensors and adopting machine learning technologies, wearable devices have recently become a prominent health care application domain. Among the related research topics in this field, one of the most important issues is detecting falls while walking. Since such falls may lead to serious injuries, automatically and promptly detecting them during daily use of smartphones and/or smart watches is a particular need. In this paper, we investigate the use of Gaussian process (GP) methods for characterizing dynamic walking patterns and detecting falls while walking with built-in wearable sensors in smartphones and/or smartwatches. For the task of characterizing dynamic walking patterns in a low-dimensional latent feature space, we propose a novel approach called auto-encoded Gaussian process dynamical model, in which we combine a GP-based state space modeling method with a nonlinear dimensionality reduction method in a unique manner. The Gaussian process methods are fit for this task because one of the most import strengths of the Gaussian process methods is its capability of handling uncertainty in the model parameters. Also for detecting falls while walking, we propose to recycle the latent samples generated in training the auto-encoded Gaussian process dynamical model for GP-based novelty detection, which can lead to an efficient and seamless solution to the detection task. Experimental results show that the combined use of these GP-based methods can yield promising results for characterizing dynamic walking patterns and detecting falls while walking with the wearable sensors.

Respiratory motion compensation for the robot-guided laser osteotome.

The use of a robot-guided laser osteotome for median sternotomy is impeded by prohibiting cutting inaccuracies due to respiration-induced motions of the thorax. With this paper, we advance today's methodologies in sternotomy procedures by introducing the concept of novel 3D functional cuts and a respiratory motion compensation algorithm for the computer-assisted and robot-guided laser osteotome, CARLO®. We present a trajectory planning algorithm for performing 3D functional cuts at a constant cutting velocity. In addition, we propose the use of Gaussian process (GP) prediction in order to anticipate the sternum's pose providing enough time for the CARLO® device to adjust the position of the laser source. We analysed the performance of the proposed algorithms on a computer-based simulation framework of the CARLO® device. The median position error of the laser focal point has shown to be reduced from 0.22mm without GP prediction to 0.19mm with GP prediction. The encouraging simulation results support the proposed respiratory motion compensation algorithm for robot-guided laser osteotomy on the thorax. Successful compensation of the respiration-induced motion of the thorax opens doors for robot-guided laser sternotomy and the related novel cutting patterns. These functional cuts hold great potential to significantly improve postoperative sternal stability and therefore reduce pain and recovery time for the patient. By enabling functional cuts, we approach an important threshold moment in the history of osteotomy, creating innovative opportunities which reach far beyond the classic linear cutting patterns.