Clustering-Guided Gp-Ucb for Bayesian Optimization

Physics-aware Gaussian processes in remote sensing

Wind turbines are growing bigger to becomemore cost-efficient. This does increase the severity of the vibrations that are present in the turbine blades, both due to predictable effects like wind shear and tower shadow, and due to less predictable effects like turbulence and flutter. If wind turbines are to become bigger and more cost-efficient, these vibrations need to be reduced. This can be done by installing trailing-edge flaps to the blades. Because of the variety of circumstances which the turbine should operate in, this results in large uncertainties. As such, we need methods that can take stochastic effects into account. Preferably we develop an algorithmthat can learn from online data how the flaps affect the wind turbine and how to optimally control them. A simple prior analysis can be done using a linearized version of the system. In this case it is important to know not only the expected cost (damage) that will be incurred by the wind turbine in various situations, but also the spread of this cost. This can for instance be done by looking at the variance of the cost function. Various expressions are available to analytically calculate this variance. Alternatively, we can prescribe a degree of stability for the system. Due to the limitations of linear approximations of systems, it is more effective to apply nonlinear regression methods. A promising one is Gaussian Process (GP) regression. Given a training set (X, y) it can predict function values f (x¤) for test points x¤. It has its basis in Bayesian probability theory, which allows it to not only make this prediction, but also give information (the variance) about its accuracy. The usual way in which GP regression is applied has a few important limitations. Most importantly, it is computationally intensive, especially when applied to constantly growing data sets. In addition, it has difficulties dealing with noise present in the training input points x. There are methods to solve either of these issues, but these tricks generally do not work well together, or their combination requires many computational resources. However, by making the right approximations, like Taylor expansions and at times even linearizations, Gaussian process regression can be applied efficiently, in an online way, to data sets with noisy input points. This enables GP regression to be used for system identification problems like online non-linear black-box modeling. Another limitation is that it can be difficult to find the optimum of a Gaussian process. The reason is that the optimum of a Gaussian process is not a fixed point but a random variable. The distribution of this optimum cannot be calculated analytically, but we can use particle methods to approximate it. We can subsequently use this principle to efficiently explore an unknown nonlinear function, trying to locate its optimum. To do so, we sample a point x from the optimum distribution, measure what the function value f (x) at this point is, update the Gaussian process approximation of the function, update the optimum distribution and repeat this process until the distribution has converged. Finding the optimum of a function like this has shown to have competitive performance at keeping the cumulative regret low, compared to similar algorithms. In addition, it allows wind turbines to tune the gains of a fixed-structure controller so as to optimize a nonlinear cost function like the damage equivalent load. All these improvements are a step forward in the application of Gaussian process regression to wind turbine applications. But as is always the case with research, there are still many things left to improve further.

Daily evaporative loss (Ep) forecasting models are decisive tools with potential applications in hydrology, the design of water systems, urban water assessments, and irrigation management. This paper performs a case study for forecasting daily Ep for Brisbane city using least-square support-vector regression (LSSVR). A limited set of predictor data with solar radiation and exposure, maximum/minimum temperatures, wind speed, and precipitation (March 1, 2014 to March 31, 2015) is adopted to develop the predictive model. The results are evaluated with Gaussian process regression (GPR), minimax probability machine regression (MPMR), and genetic programming (GP) models. In the testing phase, a correlation coefficient of 0.895 is attained between the observed and forecasted Ep by LSSVR that contrasted 0.875 (GPR), 0.864 (MPMR), and 0.628 (GP). A sensitivity test of predictor variables shows that approximately 28.5% of features are extracted from solar radiation data with 18.1% (wind speed), 16.6% (precipitation), and 10–15% (minimum and maximum temperature). The root-mean square error for LSSVR is lower than the GPR, MPMR, and GP models by 16.2, 11.4, and 79.4%, and the cumulative frequency of forecasting error attained for LSSVR is the highest within the smallest error band. The results confirm the better utility of LSSVR in relation to GP, GPR, and MPMR models for forecasting daily evaporative loss.

Detecting small planets via the radial velocity (RV) method remains challenged by signals induced by stellar variability, versus the effects of the planet(s). Here, we explore using Gaussian process (GP) regression with Transiting Exoplanet Survey Satellite (TESS) photometry in modeling RVs to help to mitigate stellar jitter from oscillations and granulation for exoplanet detection. We applied GP regression to simultaneous TESS photometric and RV data of HD 5562, a G-type subgiant ( M ⋆ = 1.09 M ⊙ , R ⋆ = 1.88 R ⊙ ) with a V magnitude of 7.17, using photometry to inform the priors for RV fitting. The RV data is obtained by the Magellan Planet Finder Spectrograph. The photometry-informed GP regression reduced the RV scatter of HD 5562 from 2.03 to 0.51 m s −1 . We performed injection and recovery tests to evaluate the potential of GPs for discovering small exoplanets around evolved stars, which demonstrate that the GP provides comparable noise reduction to the binning method. We also found that the necessity of photometric data depends on the quality of the RV dataset. For long baseline and high-cadence RV observations, GP regression can effectively mitigate stellar jitter without photometric data. However, for intermittent RV observations, incorporating photometric data improves GP fitting and enhances detection capabilities.

This paper evaluates the potential of two machine learning approaches i.e. Support vector machine (SVR) and Gaussian processes (GP) regression to model the oblique load capacity of batter pile groups. Linear regression was used to compare the performance of both SVR and GP based regression approaches to model the oblique load. Data set used consists of 147 samples obtained from the laboratory experiments. Out of the total sample size, 105 randomly selected samples were used for training whereas remaining 42 were used for testing the models. Input data set consist of angle of oblique load, pile length, sand relative density, number of vertical piles, number of batter piles where as oblique load was considered as output. Two kernel functions i.e. Polynomial and radial based kernel function were used with both SVR and GP regression. A comparison of results suggest that radial basis function based SVR approach works well in comparison to GP and linear regression based approaches and it could successfully be employed in modelling the oblique load capacity of batter pile groups. Parametric analysis and sensitivity analysis suggest that loading angle, pile length and number of batter pile were important in prediction of oblique load.

The applicability of Gaussian process (GP) regression as modeling tool within a genetic algorithm (GA) antenna optimization scheme is demonstrated. GP regression has recently been shown to be an efficient alternative to artificial neural networks (ANNs) when many accurate and fast estimates of antenna performance parameters, usually only obtainable from computationally intensive full-wave analyses, are required. GP regression has the important advantage that far fewer free parameters need to be determined; it furthermore is not subject to the implementation issues associated with neural nets. By using genetic algorithm optimization, two kinds of CPW-fed slot antennas were designed to operate within pre-specified frequency bands: an ultrawide-band (UWB) slot antenna with U-shaped tuning stub, and a dual-band slot loop antenna with separate T-shaped and insert tuning stubs. In both cases, the associated modeling problems were challenging, involving highly nonlinear mappings of several tunable geometry variables and frequency to S 11. This was especially so for the dual-band antenna, which had to operate in precisely positioned, comparatively narrow (compared to the UWB case) frequency bands (GSM and DECT) delineated by rapid changes in |S 11|. Despite using relatively few training data points given the dimensionality of the input spaces (a highly desirable model feature), the respective Gaussian process models were sufficiently accurate for the GA optimizer to yield antenna geometries that met the bandwidth specifications with ease, as was confirmed by comparison to moment-method-based simulations.

Mechanical property prediction plays a crucial role in the steel industry, enabling better materials selection and enhancing production efficiency. In this study, we propose a novel framework that facilitates optimal materials selection for steel’s alloying elements, based on accurate predictions of the Jominy hardness curve. Leveraging Gaussian Process (GP) regression, we provide probabilistic predictions of steel hardenability characteristics from alloy element composition. Taking it a step further, our framework incorporates these accurate predictions into a constrained optimization process, yielding optimal compositions that reduce overall spending while meeting performance specifications. Through data obtained from 1080 steel samples, our GP regression model exhibits high accuracy, achieving an RMSE of 1.37 and showcasing significant improvements in the field. Moreover, our constrained optimization utilizing the GP model and historical market data reveals an average cost reduction of 18% on alloying element expenses, highlighting the tangible cost-saving potential of this approach. By leveraging Gaussian Process (GP) regression, we not only achieve accurate predictions of the Jominy hardness curve based on alloy element composition, but we also introduce a crucial element of uncertainty quantification. This empowers us to place trust in the results of our optimization process, ensuring robust and reliable materials selection. The integration of GP regression and optimization provides a powerful tool for achieving cost-effective materials selection and marks a significant advancement compared to existing studies. This research underscores the promise of machine learning in the steel industry, demonstrating its ability to yield substantial cost savings and enhance decision-making in materials selection.

Hybrid algorithm for multi-objective optimization design of parallel manipulators

Gaussian process (GP) regression is widely used to find the extreme of a black-box function by iteratively approximating an objective function when new evaluation obtained. Such evaluation is usually made by optimising a given acquisition function. However, for non-parametric Bayesian optimisation, the extreme of the objective function is not a deterministic value, but a random variant with distribution. We call such distribution the maximum distribution which is generally non-analytical. To construct such maximum distribution, traditional GP regression method by optimising an acquisition function is computational cost as the GP model has a cubic computation of training data. Moreover, the introduction of acquisition function brings extra hyperparameters which made the optimisation more complicated. Recently, inspired by the idea of Sequential Monte Carlo method and its application in Bayesian optimisation, a Monte Carlo alike method is proposed to approximate the maximum distribution with weighted samples. Alternative to the method on GP model, we construct the maximum distribution within the framework of Student-t process (TP) which considers more uncertainties from the training data. Toy examples and real data experiment show the TP-based Monte Carlo maximum distribution has a competitive performance to the GP-based method.

Gaussian process regression for seismic fragility assessment of building portfolios

This paper examines experimental design procedures used to develop surrogates of computational models, exploring the interplay between experimental designs and approximation algorithms. We focus on two widely used approximation approaches, Gaussian process (GP) regression and nonintrusive polynomial approximation. First, we introduce algorithms for minimizing a posterior integrated variance (IVAR) design criterion for GP regression. Our formulation treats design as a continuous optimization problem that can be solved with gradient-based methods on complex input domains without resorting to greedy approximations. We show that minimizing IVAR in this way yields point sets with good interpolation properties and that it enables more accurate GP regression than designs based on entropy minimization or mutual information maximization. Second, using a Mercer kernel/eigenfunction perspective on GP regression, we identify conditions under which GP regression coincides with pseudospectral polynomial approximation. ...

Bayesian Optimization for Field-Scale Geological Carbon Storage

System identification through online sparse Gaussian process regression with input noise

Modelling pile capacity using Gaussian process regression

SummaryIn well control (production) optimization, the computational cost of conducting a full-physics flow simulation on a 3D, rich grid-based model poses a significant challenge. This challenge is exacerbated in a robust optimization (RO) setting, where flow simulation must be repeated for numerous geological realizations, rendering RO impractical for many field-scale cases. In this paper, we introduce and discuss a new optimization workflow that addresses this issue by providing computational efficiency, i.e., achieving a near-global optimum of the predefined objective function with minimal forward model (flow-simulation) evaluations. In this workflow, referred to as “Bayesian optimization (BO),” the objective function for samples of decision (control) variables is first computed using a proper design experiment. Then, given the samples, a Gaussian process regression (GPR) is trained to mimic the surface of the objective function as a surrogate model. While balancing the dilemma to select the next control variable between high mean, low uncertainty (exploitation) and low mean, high uncertainty (exploration), a new control variable is selected, and flow simulation is run for this new point. Later, the GPR is updated, given the output of the flow simulation. This process continues sequentially until the termination criteria are satisfied. To validate the workflow and obtain a better insight into the detailed steps, we first optimized a 1D problem. The workflow is then implemented for a 3D synthetic reservoir model to perform RO in a realistic field scenario (8-dimensional and 45-dimensional optimization problems). The workflow is compared with two other commonly used gradient-free algorithms in the literature: particle swarm optimization (PSO) and genetic algorithm (GA). The main contributions are (1) developing a new optimization workflow to address the computational cost of flow simulation in RO, (2) demonstrating the effectiveness of the workflow on a 3D grid-based model, (3) investigating the robustness of the workflow against randomness in initiation samples and discussing the results, and (4) comparing the workflow with other optimization algorithms, showing that it achieves same near-optimal results while requiring only a fraction of the computational time.