Use Of Gaussian Processes Research Articles

Efficient optimization procedure in non-linear fluid-structure interaction problem: Application to mainsail trimming in upwind conditions

This paper investigates the use of Gaussian processes to solve sail trimming optimization problems. The Gaussian process, used to model the dependence of the performance with the trimming parameters, is constructed from a limited number of performance estimations at carefully selected trimming points, potentially enabling the optimization of complex sail systems with multiple trimming parameters. The proposed approach is tested on a two-parameter trimming for a scaled IMOCA mainsail in upwind sailing conditions. We focus on the robustness of the proposed approach and study especially the sensitivity of the results to noise and model error in the point estimations of the performance. In particular, we contrast the optimization performed on a real physical model set in a wind tunnel with a fully non-linear numerical fluid-structure interaction model of the same experiments. For this problem with a limited number of trimming parameters, the numerical optimization was affordable and found to require a comparable amount of performance estimation as for the experimental case. The results reveal a satisfactory agreement for the numerical and experimental optimal trimming parameters, considering the inherent sources of errors and uncertainties in both numerical and experimental approaches. Sensitivity analyses have been eventually performed in the numerical optimization problem to determine the dominant source of uncertainties and characterize the robustness of the optima.

Gaussian approximations of small noise diffusions in Kullback–Leibler divergence

We study Gaussian approximations to the distribution of a diffusion. The approximations are easy to compute: they are defined by two simple ordinary differential equations for the mean and the covariance. Time correlations can also be computed via solution of a linear stochastic differential equation. We show, using the Kullback-Leibler divergence, that the approximations are accurate in the small noise regime. An analogous discrete time setting is also studied. The results provide both theoretical support for the use of Gaussian processes in the approximation of diffusions, and methodological guidance in the construction of Gaussian approximations in applications.

A machine learning approach to nonlinear modal analysis

Although linear modal analysis has proved itself to be the method of choice for the analysis of linear dynamic structures, its extension to nonlinear structures has proved to be a problem. A number of competing viewpoints on nonlinear modal analysis have emerged, each of which preserves a subset of the properties of the original linear theory. From the geometrical point of view, one can argue that the invariant manifold approach of Shaw and Pierre is the most natural generalisation. However, the Shaw–Pierre approach is rather demanding technically, depending as it does on the analytical construction of a mapping between spaces, which maps physical coordinates into invariant manifolds spanned by independent subsets of variables. The objective of the current paper is to demonstrate a data-based approach motivated by Shaw–Pierre method which exploits the idea of statistical independence to optimise a parametric form of the mapping. The approach can also be regarded as a generalisation of the Principal Orthogonal Decomposition (POD). A machine learning approach to inversion of the modal transformation is presented, based on the use of Gaussian processes, and this is equivalent to a nonlinear form of modal superposition. However, it is shown that issues can arise if the forward transformation is a polynomial and can thus have a multi-valued inverse. The overall approach is demonstrated using a number of case studies based on both simulated and experimental data.

Friends Recommendation with Rating Side Information

With the broad application of web 2.0 technology, various kinds of online social networks arise at present. The emergence of social networks not only helps the public to facilitate the sharing and communicating, but also helps them to know more new friends from cyberspace social circle. Therefore, friends recommendation becomes a critical function for various online social networks. Problem of friends recommendation is actually the problem of link prediction in essential, and vast majority of the solutions are traditionally based on the social relation, yet common interests are also the impor- tant factor of forming friendships in real life besides social relations. This article proposes a new matrix factorization based method for friends recommendation. The proposed method considers two data sources of social relations and inter- est ratings simultaneously, utilizes Gaussian kernel to capture interest relevance (interest similarity) between persons, makes use of Gaussian process to generate the users' profile vectors, and eventually achieves a recommendation method having the capability of commending friends with common interests. Experiments show that our method outperforms the art-of-the-state traditional link prediction methods using only social relations.

Convex Combination of Gaussian Processes for Bayesian Analysis of Deterministic Computer Experiments

The use of Gaussian processes is a popular approach to analyzing data from computer experiments. Combining more than one Gaussian process in a surrogate model for computer simulation could prove useful when there is uncertainty regarding the family of correlation functions, or when one wishes to characterize both global trends and finer details, all in the same model. We suggest a fully Bayesian treatment of the problem, taking advantage of MCMC sampling methods and providing point estimates and Bayesian credible intervals with a high degree of success, according to simulation results. We then provide guidelines for choosing a suitable batch sequential maximum entropy design for these models. Supplementary materials for this article are available online.

Gaussian Processes for POMDP-Based Dialogue Manager Optimization

A partially observable Markov decision process (POMDP) has been proposed as a dialog model that enables automatic optimization of the dialog policy and provides robustness to speech understanding errors. Various approximations allow such a model to be used for building real-world dialog systems. However, they require a large number of dialogs to train the dialog policy and hence they typically rely on the availability of a user simulator. They also require significant designer effort to hand-craft the policy representation. We investigate the use of Gaussian processes (GPs) in policy modeling to overcome these problems. We show that GP policy optimization can be implemented for a real world POMDP dialog manager, and in particular: 1) we examine different formulations of a GP policy to minimize variability in the learning process; 2) we find that the use of GP increases the learning rate by an order of magnitude thereby allowing learning by direct interaction with human users; and 3) we demonstrate that designer effort can be substantially reduced by basing the policy directly on the full belief space thereby avoiding ad hoc feature space modeling. Overall, the GP approach represents an important step forward towards fully automatic dialog policy optimization in real world systems.

One-class classification with Gaussian processes

Detecting instances of unknown categories is an important task for a multitude of problems such as object recognition, event detection, and defect localization. This article investigates the use of Gaussian process (GP) priors for this area of research. Focusing on the task of one-class classification, we analyze different measures derived from GP regression and approximate GP classification. We also study important theoretical connections to other approaches and discuss their underlying assumptions. Experiments are performed using a large number of datasets and different image kernel functions. Our findings show that our approaches can outperform the well-known support vector data description approach indicating the high potential of Gaussian processes for one-class classification. Furthermore, we show the suitability of our methods in the area of attribute prediction, defect localization, bacteria recognition, and background subtraction. These applications and experiments highlight the easy applicability of our method as well as its state-of-the-art performance compared to established methods.

Terrain Reconstruction of Glacial Surfaces : Robotic Surveying Techniques

The capability to monitor natural phenomena using mobile sensing is a benefit to the Earth science community, given the potentially large impact that humans have on naturally occurring processes. Such phenomena can be readily monitored using networks of mobile sensor nodes that are tasked to regions of interest by scientists. In our article, we hone in on a very specific domain, elevation changes in glacial surfaces, to demonstrate a concept applicable to any spatially distributed phenomena (e.g., temperature or humidity). Our article leverages the sensing of a vision-based odometry system and the design of robotic surveying navigation rules to reconstruct scientific areas of interest, with the goal of monitoring elevation changes in glacial regions. The reconstruction methodology presented makes use of Gaussian process (GP) regression to combine sparse visual landmarks extracted from the glacial scenery into a dense topographic map. Further, this method allows for the natural inclusion of a priori terrain knowledge, such as existing digital elevation models. Results from this system are presented from a three-dimensional (3-D) glacial simulation modeled after actual field trials on Alaskan glaciers. Additionally, we introduce a theory behind spatial coverage, in the context of sampling, as achieved by an intelligently navigating agent. Finally, we validate the output from our methodology and provide results and show that the reconstructed terrain error complies with acceptable mapping standards found in the scientific community.

A Gemini ground-based transmission spectrum of WASP-29b: a featureless spectrum from 515 to 720 nm

We report Gemini-South GMOS observations of the exoplanet system WASP-29 during primary transit as a test case for differential spectrophotometry. We use the multi-object spectrograph to observe the target star and a comparison star simultaneously to produce multiple light curves at varying wavelengths. The 'white' light curve and fifteen 'spectral' light curves are analysed to refine the system parameters and produce a transmission spectrum from 515 to 720nm. All light curves exhibit time-correlated noise, which we model using a variety of techniques. These include a simple noise rescaling, a Gaussian process model, and a wavelet based method. These methods all produce consistent results, although with different uncertainties. The precision of the transmission spectrum is improved by subtracting a common signal from all the spectral light curves, reaching a typical precision of ~1x10^-4 in transit depth. The transmission spectrum is free of spectral features, and given the non-detection of a pressure broadened Na feature, we can rule out the presence of a Na rich atmosphere free of clouds or hazes, although we cannot rule out a narrow Na core. This indicates that Na is not present in the atmosphere, and/or that clouds/hazes play a significant role in the atmosphere and mask the broad wings of the Na feature, although the former is a more likely explanation given WASP-29b's equilibrium temperature of ~970 K, at which Na can form various compounds. We also briefly discuss the use of Gaussian process and wavelet methods to account for time correlated noise in transit light curves.

An approximate inference with Gaussian process to latent functions from uncertain data

Most formulations of supervised learning are often based on the assumption that only the outputs data are uncertain. However, this assumption might be too strong for some learning tasks. This paper investigates the use of Gaussian processes to infer latent functions from a set of uncertain input–output examples. By assuming Gaussian distributions with known variances over the inputs–outputs and using the expectation of the covariance function, it is possible to analytically compute the expected covariance matrix of the data to obtain a posterior distribution over functions. The method is evaluated on a synthetic problem and on a more realistic one, which consist in learning the dynamics of a cart–pole balancing task. The results indicate an improvement of the mean squared error and the likelihood of the posterior Gaussian process when the data uncertainty is significant.

Gaussian process modeling of large‐scale terrain

AbstractBuilding a model of large‐scale terrain that can adequately handle uncertainty and incompleteness in a statistically sound way is a challenging problem. This work proposes the use of Gaussian processes as models of large‐scale terrain. The proposed model naturally provides a multiresolution representation of space, incorporates and handles uncertainties aptly, and copes with incompleteness of sensory information. Gaussian process regression techniques are applied to estimate and interpolate (to fill gaps in occluded areas) elevation information across the field. The estimates obtained are the best linear unbiased estimates for the data under consideration. A single nonstationary (neural network) Gaussian process is shown to be powerful enough to model large and complex terrain, effectively handling issues relating to discontinuous data. A local approximation method based on a “moving window” methodology and implemented using k‐dimensional (KD)‐trees is also proposed. This enables the approach to handle extremely large data sets, thereby completely addressing its scalability issues. Experiments are performed on large‐scale data sets taken from real mining applications. These data sets include sparse mine planning data, which are representative of a global positioning system–based survey, as well as dense laser scanner data taken at different mine sites. Further, extensive statistical performance evaluation and benchmarking of the technique has been performed through cross‐validation experiments. They conclude that for dense and/or flat data, the proposed approach will perform very competitively with grid‐based approaches using standard interpolation techniques and triangulated irregular networks using triangle‐based interpolation techniques; for sparse and/or complex data, however, it would significantly outperform them. © 2009 Wiley Periodicals, Inc.

Stable and Efficient Gaussian Process Calculations

The use of Gaussian processes can be an effective approach to prediction in a supervised learning environment. For large data sets, the standard Gaussian process approach requires solving very larg...

Bayesian Learning with Gaussian Processes for Supervised Classification of Hyperspectral Data

Recent advances in kernel machines promote the novel use of Gaussian processes (GP) for Bayesian learning. Our purpose is to introduce GP models into the remote sensing community for supervised learning as exemplified in this study for classifying hyperspectral images. We first provided the mathematical formulation of GP models concerning both regression and classification; described several GP classifiers (GPCLs) and the automatic learning of kernel parameters; and then, examined the effectiveness of GPCLs compared with K-nearest neighbor (KNN) and Support Vector Machines (SVM). Experiment results on an Airborne Visible/Infrared Imaging Spectroradiometer image indicate that the GPCLs outperform KNN and yield classification accuracies comparable to or even better than SVMs. This study shows that GP models, though with a larger computation scaling than SVM, bring a competitive tool for remote sensing applications related to classification or possibly regression, particularly with small or moderate sizes of training datasets.

Determination of the Sensitivity of Heat Transfer Systems Using Global Sensitivity and Gaussian Processes

A critical aspect of the design of systems or experiments is a sensitivity analysis to determine the effects of the different variables. This is usually done by representing the response by a Taylor series and evaluating the first-order derivatives at a nominal operating point. When there is uncertainty about the operating point, the common approach is the construction of a response surface and Monte Carlo sampling based on the probability distribution of these uncertain variables. Because of the expense of Monte Carlo sampling, it is important to restrict the analysis to those variables to which the response is most sensitive. Identification of the most sensitive parameters can be conveniently done using Global sensitivity, which both defines the most critical variables and also quantifies the effects of interacting variables. This also can be a computationally expensive process and, for complex models, is generally prohibitively expensive. A solution is the use of Gaussian processes that allows one to create a response surface using easy-to-evaluate functions. This paper describes the use of these ideas for a heat transfer problem.

Graph kernels and Gaussian processes for relational reinforcement learning

RRL is a relational reinforcement learning system based on Q-learning in relational state-action spaces. It aims to enable agents to learn how to act in an environment that has no natural representation as a tuple of constants. For relational reinforcement learning, the learning algorithm used to approximate the mapping between state-action pairs and their so called Q(uality)-value has to be very reliable, and it has to be able to handle the relational representation of state-action pairs. In this paper we investigate the use of Gaussian processes to approximate the Q-values of state-action pairs. In order to employ Gaussian processes in a relational setting we propose graph kernels as a covariance function between state-action pairs. The standard prediction mechanism for Gaussian processes requires a matrix inversion which can become unstable when the kernel matrix has low rank. These instabilities can be avoided by employing QR-factorization. This leads to better and more stable performance of the algorithm and a more efficient incremental update mechanism. Experiments conducted in the blocks world and with the Tetris game show that Gaussian processes with graph kernels can compete with, and often improve on, regression trees and instance based regression as a generalization algorithm for RRL.

Failure probability of reinforced concrete columns under stochastic loads

This paper deals with a technique to calculate the time-variant failure probability of reinforced concrete columns under the combined actions of bending moment and axial force when the applied loads are stationary or nonstationary stochastic processes. An outcrossing rate formulation is applied. Also the limit state surface for the structure is nonlinear. The technique presented is a simple approach suitable for checking the structural reliability of reinforced concrete columns. It avoids the use of multidimensional integration in the computation of the outcrossing rate; this is achieved by piecewise linearization of the limit state surface and the use of Gaussian processes. An example is given to illustrate the approach.