Heteroscedastic Gaussian Process Research Articles

Heteroscedastic Gaussian processes for data normalisation in probabilistic novelty detection of a wind turbine

This study investigates the data normalisation of modal parameters of an operating concrete–steel hybrid onshore wind turbine tower considering also the identification uncertainty. In order to take into account the Environmental and Operational Condition (EOC)-dependent variance, sparse heteroscedastic Gaussian processes (GPs) are used for the data normalisation. Following a typical vibration-based Structural Health Monitoring (SHM) scheme, data normalisation of the natural frequencies and the mode shapes is performed first. Subsequently, a metric is defined which takes into account both the identification uncertainty and the operation-dependent uncertainty in order to enable novelty detection.The data normalisation methods must be able to handle uncertainties of different magnitudes due to EOCs in the data. In this context, GPs can be a suitable tool. However, standard GPs assume homoscedasticity, which is an unrealistic assumption in the case of EOC-dependent variance. Using a heteroscedastic GP instead, the variance of the data is better mapped and allows comparison with the identification uncertainties of Bayesian operational modal analysis (BAYOMA), taking into account the specifics of closely spaced modes of the tower structure. This leads to a better interpretation of the data and enables the introduction of a probabilistic novelty metric.This data normalisation approach, taking into account EOC-dependent uncertainties using heteroscedastic GPs, is being applied for the first time to a tower of a full scale 3.4 MW wind turbine in operation. Following this approach, it is possible to detect smaller changes in natural frequencies and second-order modal assurance criterion (S2MAC) compared to the assumption of homoscedasticity within the GP. In addition, a novelty was detected using the S2MAC during the period under study. Therefore, it can be illustrated that mode shape-based metrics tend to be more sensitive than purely frequency-based ones. However, it is difficult to assess the significance of such changes for structural integrity without further information.

High Spatial Resolution Leaf Area Index Estimation for Woodland in Saihanba Forestry Center, China

Owing to advancements in satellite remote sensing technology, the acquisition of global land surface parameters, notably, the leaf area index (LAI), has become increasingly accessible. The Sentinel-2 (S2) satellite plays an important role in the monitoring of ecological environments and resource management. The prevalent use of the 20 m spatial resolution band in S2-based inversion models imposes significant limitations on the applicability of S2 data in applications requiring finer spatial resolution. Furthermore, although a substantial body of research on LAI retrieval using S2 data concentrates on agricultural landscapes, studies dedicated to forest ecosystems, although increasing, remain relatively less prevalent. This study aims to establish a viable methodology for retrieving 10 m resolution LAI data in forested regions. The empirical model of the soil adjusted vegetation index (SAVI), the backpack neural network based on simulated annealing (SA-BP) algorithm, and the variational heteroscedastic Gaussian process regression (VHGPR) model are established in this experiment based on the LAI data measured and the corresponding 10 m spatial resolution S2 satellite surface reflectance data in the Saihanba Forestry Center (SFC). The LAI retrieval performance of the three models is then validated using field data, and the error sources of the best performing VHGPR models (R2 of 0.8696 and RMSE of 0.5078) are further analyzed. Moreover, the VHGPR model stands out for its capacity to quantify the uncertainty in LAI estimation, presenting a notable advantage in assessing the significance of input data, eliminating redundant bands, and being well suited for uncertainty estimation. This feature is particularly valuable in generating accurate LAI products, especially in regions characterized by diverse forest compositions.

A Nuclear Data Evaluation Pipeline for the Fast Neutron Energy Range – using heteroscedastic Gaussian processes to treat model defects

A Nuclear Data Evaluation Pipeline for the Fast Neutron Energy Range – using heteroscedastic Gaussian processes to treat model defects

Adaptive active subspace-based metamodeling for high-dimensional reliability analysis

To address the challenges of reliability analysis in high-dimensional probability spaces, this paper proposes a new metamodeling method that couples active subspace, heteroscedastic Gaussian process, and active learning. The active subspace is leveraged to identify low-dimensional salient features of a high-dimensional computational model. A surrogate computational model is built in the low-dimensional feature space by a heteroscedastic Gaussian process. Active learning adaptively guides the surrogate model training toward the critical region that significantly contributes to the failure probability. A critical trait of the proposed method is that the three main ingredients–active subspace, heteroscedastic Gaussian process, and active learning–are coupled to adaptively optimize the feature space mapping in conjunction with the surrogate modeling. This coupling empowers the proposed method to accurately solve nontrivial high-dimensional reliability problems via low-dimensional surrogate modeling. Finally, numerical examples of a high-dimensional nonlinear function and structural engineering applications are investigated to verify the performance of the proposed method.

Extended Target Tracking in Human–Robot Coexisting Environments via Multisensor Information Fusion: A Heteroscedastic Gaussian Process Regression-Based Approach

In this article, a new systematic approach to sensor fusion and state estimation is proposed for extended target tracking in human–robot coexisting environments. The developed method, called <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">human feature-based extended target tracking via multisensor information fusion</i> (HFBETT-MSIF), can assimilate information from the onboard camera and sonar sensor of a mobile robot in a unified way, during tracking of a pair of human shoes. A novel generalized measurement model containing the complete information of the human target is formulated for both sensors, thus rendering the tracking system potentially robust to the failure of any one sensor. The study illustrates how <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">heteroscedastic Gaussian process</i> (HGP) regression can be used to derive the measurement model. It also develops an advanced HGP model, called <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">bias-minimized most likely HGP</i> , to interpret the real-world shoe-contour data subjected to heteroscedastic noise. Performance evaluations conducted for real-life shoe tracking demonstrate the supremacy of the HFBETT-MSIF.

Multilevel emulation for stochastic computer models with application to large offshore wind farms

AbstractRenewable energy projects, such as large offshore wind farms, are critical to achieving low-emission targets set by governments. Stochastic computer models allow us to explore future scenarios to aid decision making while considering the most relevant uncertainties. Complex stochastic computer models can be prohibitively slow, and thus an emulator may be constructed and deployed to allow for efficient computation. We present a novel heteroscedastic Gaussian Process emulator that exploits cheap approximations to a stochastic offshore wind farm simulator. We also conduct a probabilistic sensitivity analysis to understand the influence of key parameters in the wind farm model, which will help us to plan a probability elicitation in the future.

Physically meaningful uncertainty quantification in probabilistic wind turbine power curve models as a damage-sensitive feature

A wind turbines’ power curve is an easily accessible form of damage-sensitive data, and as such is a key part of structural health monitoring (SHM) in wind turbines. Power curve models can be constructed in a number of ways, but the authors argue that probabilistic methods carry inherent benefits in this use case, such as uncertainty quantification and allowing uncertainty propagation analysis. Many probabilistic power curve models have a key limitation in that they are not physically meaningful – they return mean and uncertainty predictions outside of what is physically possible (the maximum and minimum power outputs of the wind turbine). This paper investigates the use of two bounded Gaussian processes (GPs) in order to produce physically meaningful probabilistic power curve models. The first model investigated was a warped heteroscedastic Gaussian process, and was found to be ineffective due to specific shortcomings of the GP in relation to the warping function. The second model – an approximated GP with a Beta likelihood was highly successful and demonstrated that a working bounded probabilistic model results in better predictive uncertainty than a corresponding unbounded one without meaningful loss in predictive accuracy. Such a bounded model thus offers increased accuracy for performance monitoring and increased operator confidence in the model due to guaranteed physical plausibility.

Machine learning for additive manufacturing: Predicting materials characteristics and their uncertainty

Additive manufacturing (AM) is known for versatile fabrication of complex parts, while also allowing the synthesis of materials with desired microstructures and resulting properties. These benefits come at a cost: process control to manufacture parts within given specifications is very challenging due to the relevance of a large number of processing parameters. Efficient predictive machine learning (ML) models trained on small datasets, can minimize this cost. They also allow to assess the quality of the dataset inclusive of uncertainty. This is important in order for additively manufactured parts to meet property specifications not only on average, but also within a given variance or uncertainty. Here, we demonstrate this strategy by developing a heteroscedastic Gaussian process (HGP) model, from a dataset based on laser powder bed fusion of a glass-forming alloy at varying processing parameters. Using amorphicity as the microstructural descriptor, we train the model on our Zr52.5Cu17.9Ni14.6Al10Ti5 (at.%) alloy dataset. The HGP model not only accurately predicts the mean value of amorphicity, but also provides the respective uncertainty. The quantification of the aleatoric and epistemic uncertainty contributions allows to assess intrinsic inaccuracies of the dataset, as well as identify underlying physical phenomena. This HGP model approach enables to systematically improve ML-driven AM processes.

Probabilistic Conflict Detection Using Heteroscedastic Gaussian Process and Bayesian Optimization

Probabilistic Conflict Detection Using Heteroscedastic Gaussian Process and Bayesian Optimization

Performance Prediction of NMPC Algorithms with Incomplete Optimization

Nonlinear model predictive control (NMPC) in real-time applications leads to a trade-off between computation time and solution accuracy, resulting in uncertainty about the control quality. This paper faces this issue by monitoring and predicting the cost of the MPC in every optimization step. A heteroscedastic Gaussian process regression method is developed that can be used to predict the number of optimization steps necessary for a sufficient cost decrease. The method is demonstrated on an example.

Gaussian process metamodels for floating offshore wind turbine platforms

The dynamics of floating offshore wind turbines (FOWT) are quite complex as it involves interactions in multiple domains. The dynamic response of FOWT is numerically evaluated using coupled aero-hydro-servo-elastic analysis. In the design stages, the dynamic analysis of FOWT is carried out for different environmental conditions. These analyses can be time-consuming and computationally expensive. This paper proposes the use of metamodels based on the Gaussian process for representing the hydrodynamics and structural dynamics of the floating platform. The input to Gaussian process metamodels is taken as wave elevation, forces and moments at the interface of the tower and platform. The output of the metamodels is the displacements (surge, sway and heave) and the rotations (roll, pitch and yaw) of the platform. Three different Gaussian process metamodels are considered in this paper — Standard Gaussian Process, Variational Heteroscedastic Gaussian Process and Sparse Spectrum Gaussian Process. The construction of the metamodels is demonstrated for an NREL 5MW turbine supported on an ITI barge platform. The performance of the Gaussian metamodels is compared in terms of evaluation indices, convergence, residuals and computational efficiency. The sensitivity analysis of the metamodels is carried out to verify that the interdependence between the variables is captured accurately. All three Gaussian process metamodels are found to be effective in emulating the response of the platform. These models can be effectively used in situations where simple and computationally inexpensive models are required.

Estimation of first-passage probability under stochastic wind excitations by active-learning-based heteroscedastic Gaussian process

In the processes for designing and assessing structural systems, it is essential to evaluate their reliability against stochastic loads caused by natural or human-made hazards, e.g., wind loads, earthquakes, and collisions. The first-passage probability is a crucial measure of a system's reliability under such conditions. However, the first-passage probability estimation generally requires repetitive dynamic simulations and thus may result in high computational cost. This paper proposes a new active-learning-based surrogate method that efficiently estimates the first-passage probability under stochastic wind excitations to address the computational issue. The proposed method introduces an alternative formulation using the conditional distribution of the maximum response given time-invariant uncertain parameters to handle the high dimensionality of stochastic excitation sequences. The method employs the Gaussian-process-based surrogate model with heteroscedastic noises to fit the distribution parameter functions considering uncertainties arising from the structural system and the environmental loads. In addition, an adaptive training process of surrogates is introduced to identify the best experimental designs achieving efficient convergence. The numerical examples of an eight-story building and a transmission tower demonstrate that the proposed method can produce accurate estimation results with fewer structural simulations than existing methods.

Estimating soil moisture content under grassland with hyperspectral data using radiative transfer modelling and machine learning.

The monitoring of soil moisture content (SMC) at very high spatial resolution (<10m) using unmanned aerial systems (UAS) is of high interest for precision agriculture and the validation of large scale SMC products. Data-driven approaches are the most common method to retrieve SMC with UAS-borne data at water limited sites over non-disturbed agricultural crops. A major disadvantage of data-driven algorithms is the limited transferability in space and time and the need of a high number of ground reference samples. Physically-based approaches are less dependent on the amount of samples and are transferable in space and time. This study explores the potential of (1) a hybrid method targeting the soil brightness factor of the PROSAIL model using a variational heteroscedastic Gaussian Processes regression (VHGPR) algorithm, and (2) a data-driven method employing VHGPR for the retrieval of SMC over three grassland sites based on UAS-borne VIS-NIR (399-1001 nm) hyperspectral data. The sites were managed by mowing (Fendt), grazing (Grosses Bruch) and irrigation (Marquardt). With these distinct local pre-conditions we aimed to identify factors that favor and limit the retrieval of SMC. The hybrid approach presented encouraging results in Marquardt (RMSE = 1.5 Vol_%, R2 = 0.2). At the permanent grassland sites (Fendt, Grosses Bruch) the thatch layer jeopardized the application of the hybrid model. We identified the complex canopy structure of grassland as the main factor impacting the hybrid SMC retrieval. The data-driven approach showed high accuracy for Fendt (R2 = 0.84, RMSE = 8.66) and Marquardt (R2 = 0.4, RMSE = 10.52). All data-driven models build on the LAI-SMC relationship. However, this relationship was hampered by mowing (Fendt), leading to a lack of transferability in time. The alteration of plant traits by grazing prevents finding a relationship with SMC in Grosses Bruch. In Marquardt, we identified the timelag between changes in SMC and plant response as the main reason of decrease in model accuracy. Yet, the model performance is accurate in undisturbed and water-limited areas (Marquardt). The analysis points to challenges that need to be tackled in future research and opens the discussion for the development of robust models to retrieve high resolution SMC from UAS-borne remote sensing observations.

Multiagent coverage search based on Voronoi and sparse heteroscedastic Gaussian process

AbstractThis work is concerned with multiagent cooperative and autonomous coverage search in an unknown environment. A novel Voronoi and sparse heteroscedastic Gaussian process (SHGP)‐based search method is proposed to improve the target search efficiency. By means of heteroscedastic Gaussian process (HGP), each agent can estimate the probability of target existence in given locations online according to the local probability map. Then, these local probability maps are fused together to obtain a posterior probability distribution for target prediction. Based on that, the agents' search paths within Voronoi cells will be adjusted adaptively online to make sure all the agents can search for targets more efficiently. A challenging issue then falls in the computational complexity of the HGP. To solve this problem, we propose an adaptive ranked‐set sampling algorithm based on the weighted sampling process and K‐means clustering algorithm. Theoretical analysis on the discrepancy of the kernel matrix between the original probability points and the sampling points is also given. Simulation results demonstrate that our proposed coverage search method can improve the performance of the area search compared with the existing Voronoi‐based one.

Probabilistic surrogate modeling of offshore wind-turbine loads with chained Gaussian processes

Heteroscedastic Gaussian process regression, based on the concept of chained Gaussian processes, is used to build surrogates to predict site-specific loads on an offshore wind turbine. Stochasticity in the inflow turbulence and irregular waves results in load responses that are best represented as random variables rather than deterministic values. Moreover, the effect of these stochastic sources on the loads depends strongly on the mean environmental conditions - for instance, at low mean wind speeds, inflow turbulence produces much less variability in loads than at high wind speeds. Statistically, this is known as heteroscedasticity. Deterministic and most stochastic surrogates do not account for the heteroscedastic noise, giving an incomplete and potentially misleading picture of the structural response. In this paper, we draw on the recent advancements in statistical inference to train a heteroscedastic surrogate model on a noisy database to predict the conditional pdf of the response. The model is informed via 10-minute load statistics of the IEA-10MW-RWT subject to both aero- and hydrodynamic loads, simulated with OpenFAST. Its performance is assessed against the standard Gaussian process regression. The predicted mean is similar in both models, but the heteroscedastic surrogate approximates the large-scale variance of the responses significantly better.

A new GW-based heteroscedastic Gaussian process method for online crack evaluation

Accurate evaluation of fatigue crack by using structural health monitoring (SHM) methods is very important to the life management of aircraft structures. However, fatigue crack propagation is always a complicated process for individual aircraft structures during their long-term service. And the monitoring has to be performed under different environmental and operational conditions. These factors result in the uncertain distribution of the damage index obtained by SHM. The distribution usually changes along with the service time, which can be defined as heteroscedasticity. The heteroscedasticity characteristic of the uncertain distribution of damage index has an important negative impact on the SHM based diagnostics algorithms if not considered during the evaluation. However, till now, few researchers have considered this important aspect. To improve the evaluation accuracy under different environmental and operational conditions for individual aircraft, this paper proposes a new guided wave-based heteroscedastic Gaussian process method. Gaussian process quantile regression is adopted to estimate the conditional quantiles of the damage index distribution during the service to deal with the heteroscedasticity. The method is validated on an attachment lug fatigue test, an important aircraft structural component. The experimental results demonstrate that the proposed method can quantify the heteroscedastic uncertainty associated and obviously improve the quality of crack evaluation. For the serious heteroscedastic specimen, the maximum evaluation is only 0.7 mm reduced from the original 7.4 mm, which is an order of magnitude.

Heteroscedastic sparse Gaussian process regression-based stochastic material model for plastic structural analysis

Describing the material flow stress and the associated uncertainty is essential for the plastic stochastic structural analysis. In this context, a data-driven approach-heteroscedastic sparse Gaussian process regression (HSGPR) with enhanced efficiency is introduced to model the material flow stress. Different from other machine learning approaches, e.g. artificial neural network (ANN), which only estimate the deterministic flow stress, the HSGPR model can capture the flow stress and its uncertainty simultaneously from the dataset. For validating the proposed model, the experimental data of the Al 6061 alloy is used here. Without setting a priori assumption on the mathematical expression, the proposed HSGPR-based flow stress model can produce a better prediction of the experimental stress data than the ANN model, the conventional GPR model, and Johnson Cook model at elevated temperatures. After the HSGPR-based flow stress model is implemented into finite element analysis, two numerical examples with synthetic material properties are performed to demonstrate the model’s capability in stochastic plastic structural analysis. The results have shown that with sufficient data, the distribution of the structural load carrying capacity at elevated temperatures and the variation of load–displacement curves during the loading and unloading processes can be accurately predicted by the HSGPR-based flow stress model.

Safe Corridor Learning For Model Predictive Path Following Control

Safe collision-free operation of autonomous systems, such as mobile robots in crowded, uncertain, only partially known environments, is challenging. We propose learning a collision-free corridor from demonstration via heteroscedastic Gaussian processes. We incorporate available deterministic obstacle information in the learning procedure to derive safety guarantees for the corridor. The learned passage is utilized in a model predictive path planning controller that steers the system safely through the environment. The achievable results are underlined in simulations considering a mobile robot.

Achieving robustness to aleatoric uncertainty with heteroscedastic Bayesian optimisation

Bayesian optimisation is a sample-efficient search methodology that holds great promise for accelerating drug and materials discovery programs. A frequently-overlooked modelling consideration in Bayesian optimisation strategies however, is the representation of heteroscedastic aleatoric uncertainty. In many practical applications it is desirable to identify inputs with low aleatoric noise, an example of which might be a material composition which displays robust properties in response to a noisy fabrication process. In this paper, we propose a heteroscedastic Bayesian optimisation scheme capable of representing and minimising aleatoric noise across the input space. Our scheme employs a heteroscedastic Gaussian process surrogate model in conjunction with two straightforward adaptations of existing acquisition functions. First, we extend the augmented expected improvement heuristic to the heteroscedastic setting and second, we introduce the aleatoric noise-penalised expected improvement (ANPEI) heuristic. Both methodologies are capable of penalising aleatoric noise in the suggestions. In particular, the ANPEI acquisition yields improved performance relative to homoscedastic Bayesian optimisation and random sampling on toy problems as well as on two real-world scientific datasets. Code is available at: https://github.com/Ryan-Rhys/Heteroscedastic-BO

Modelling and forecasting of SHM strain measurement for a large-scale suspension bridge during typhoon events using variational heteroscedastic Gaussian process

The modelling and forecasting (M&F) of strain measurement (as a kind of local structural responses) during typhoon events provides valuable insight into the structural condition assessment of large suspension bridges. However, the presence of time-dependent noise in reality can pose difficulties for forecasting the field data obtained by structural health monitoring (SHM) systems. Gaussian process regression (GPR), as a nonparametric model, can obtain probabilistic estimation outputs, but its constant noise assumption hampers the reliability of the forecasting model. In this study, Variational Heteroscedastic Gaussian Process (VHGP), a combination of variational approximation and heteroscedastic Gaussian process (HGP), is applied to perform modelling and forecasting for SHM strain field data during typhoon events because of its heteroskedasticity characteristics, higher forecasting accuracy and strong ability to quantify uncertainty. The proposed M&F method is exemplified by using SHM monitoring strain data acquired from the instrumented Tsing Ma Suspension Bridge during typhoon events. The results reveal that VHGP has a better regression accuracy and can obtain varying confidence intervals which reflect noise variations. Meanwhile, VHGP yields more robust forecasting results. The uncertainty analysis shows that VHGP is competent to evaluate the noise level change of strain responses brought by typhoons, providing a basis for conducting structural health condition assessment for large-scale bridges.