Nonstationary Gaussian Process Research Articles

Surrogate modeling with non-stationary-noise based Gaussian process regression and K-Fold ANN for systems featuring uneven sensitivity distribution

In the realm of aircraft design, there is a prevalent need for surrogate modeling techniques capable of efficiently and accurately modeling objects with high evaluation costs and uneven sensitivity distributions, such as those encountered in the nonlinear buckling of thin-walled structures and aerodynamics at transonic speeds. However, the non-stationary Gaussian Process Regression (GPR) model requires extensive hyperparameters or judicious assumptions, limiting its applications in such tasks, and the commonly used K-Fold methodology suffers from inherent instability. Addressing these issues, this paper proposes an active learning method for surrogate modeling in systems featuring uneven sensitivity distribution, integrating non-stationary-noise GPR and K-Fold Artificial Neural Network (ANN) methodology, namely NSN-GPR-KF. Instead of relying on a non-stationary kernel function, the algorithm regards noise as a non-stationary factor within a stationary GPR framework, with noise levels estimated via the K-Fold ANN methodology, allowing GPR to efficiently adapt to non-stationary systems while avoiding the aforementioned requirements for non-stationary GPR. Case validations, performed on two test functions with pronounced uneven spatial sensitivity, demonstrate that NSN-GPR-KF outperforms commonly used stationary GPR and pure K-Fold methodology. It synthesizes the exploration capabilities of GPR with the exploitation proficiency of the K-Fold methodology, achieving comparable global accuracy with reduced sample sizes—up to 26% and 22% less than the other two algorithms, respectively. The algorithm was successfully applied to predict the buckling load of thin-walled pipe beams, demonstrating accelerated convergence and approximately 25% greater accuracy compared to pure K-Fold model. These results suggest that NSN-GPR-KF can serve as an efficient and precise modeling tool for engineering tasks with high evaluation costs and complex sensitivity distributions.

The higher-order analysis of response for linear structure under non-Gaussian stochastic excitations with known statistical moments and power spectral density

The frequency-domain analysis method is a fundamental component of the random vibration analysis, in which the corresponding moment spectrum of excitations are the prerequisite. Nevertheless, the determination of the higher-order moment spectra for non-Gaussian stochastic excitations continues to pose a significant challenge in the existing research works. This paper introduces an accurate and efficient computational approach for determining higher-order moment spectra of non-Gaussian stochastic excitations with known statistical moments and power spectral density (PSD). Firstly, following the idea of the simulation method of non-Gaussian stochastic processes, the transformation model between the stationary non-Gaussian stochastic process and underlying Gaussian stochastic process is determined based on the known statistical moments, the PSD of the underlying Gaussian stochastic process is determined using the transformation model. Secondly, the approximate higher-order moment spectra models of stationary non-Gaussian stochastic processes are presented by the PSD of the underlying Gaussian process. Subsequently, the higher-order moment spectra models of non-Gaussian excitations are utilized to compute the higher-order moment spectra of response for the linear structure via the auxiliary harmonic excitation generalized method. Finally, three numerical examples are examined to assess the efficiency and accuracy of the proposed approximate models for the higher-order moment spectra of non-Gaussian stochastic processes.

Conditional simulation of stationary non-Gaussian processes based on unified hermite polynomial model

The conditional simulation of non-Gaussian excitations utilizing records from the monitoring system is of great significance for hazard mitigation. To this end, this paper proposes a novel conditional non-Gaussian simulation method. In this method, the Unified Hermite Polynomial Model (UHPM) is used to describe the transformation relationship between recorded and unrecorded non-Gaussian processes and their underlying Gaussian counterparts. Meanwhile, an explicit transformation model between their correlation functions is also provided. Then, the covariance matrix of Fourier coefficients of the underlying Gaussian processes is constructed. Based on this covariance matrix, the conditional samples of Fourier coefficients are generated and substituted into the Spectral Representation Method (SRM) to perform the conditional simulation of the underlying Gaussian processes. Finally, the conditionally simulated samples of the underlying Gaussian processes are transformed into the non-Gaussian samples by the UHPM. To showcase the precision and efficacy of the proposed method, two numerical examples involving the conditional simulations of non-Gaussian ground motions and non-Gaussian wind pressure coefficients are provided.

Modelling Irrational Behaviour of Residential End Users Using Non-Stationary Gaussian Processes

Modelling Irrational Behaviour of Residential End Users Using Non-Stationary Gaussian Processes

Probabilistic signal estimation for vibrational spectroscopy with a flexible non-stationary Gaussian process baseline model

Vibrational spectroscopy techniques enable accurate chemical detection and quantification, but the extraction of spectral peak parameters is frequently hampered by an underlying baseline. Because the signal and baseline are additive, it is difficult to distinguish between signal peaks and baseline effects when the baseline is not smooth. Using surface enhanced Raman spectroscopy (SERS) and near-infrared (NIR) spectroscopy as examples, we show how to estimate the signal and the baseline jointly while imposing a high-capacity non-stationary Gaussian process on the baseline. This allows us to both obtain accurate estimation and meaningful uncertainty estimates on interpretable peak parameters. We demonstrate this on artificially generated SERS maps, a challenging real-world SERS case, and a benchmark NIR dataset.

High-dimensional modeling of spatial and spatio-temporal conditional extremes using INLA and Gaussian Markov random fields

The conditional extremes framework allows for event-based stochastic modeling of dependent extremes, and has recently been extended to spatial and spatio-temporal settings. After standardizing the marginal distributions and applying an appropriate linear normalization, certain non-stationary Gaussian processes can be used as asymptotically-motivated models for the process conditioned on threshold exceedances at a fixed reference location and time. In this work, we adapt existing conditional extremes models to allow for the handling of large spatial datasets. This involves specifying the model for spatial observations at d locations in terms of a latent mll d dimensional Gaussian model, whose structure is specified by a Gaussian Markov random field. We perform Bayesian inference for such models for datasets containing thousands of observation locations using the integrated nested Laplace approximation, or INLA. We explain how constraints on the spatial and spatio-temporal Gaussian processes, arising from the conditioning mechanism, can be implemented through the latent variable approach without losing the computationally convenient Markov property. We discuss tools for the comparison of models via their posterior distributions, and illustrate the flexibility of the approach with gridded Red Sea surface temperature data at over 6,000 observed locations. Posterior sampling is exploited to study the probability distribution of cluster functionals of spatial and spatio-temporal extreme episodes.

Scalable computations for nonstationary Gaussian processes

Scalable computations for nonstationary Gaussian processes

On Nonstationary Gaussian Process Model for Solving Data-Driven Optimization Problems.

In data-driven evolutionary optimization, most existing Gaussian processes (GPs)-assisted evolutionary algorithms (EAs) adopt stationary GPs (SGPs) as surrogate models, which might be insufficient for solving most optimization problems. This article finds that GPs in the optimization problems are nonstationary with great probability. We propose to employ a nonstationary GP (NSGP) surrogate model for data-driven evolutionary optimization, where the mean of the NSGP is allowed to vary with the decision variables, while its residue variance follows an SGP. In this article, the nonstationarity of GPs in the tested functions is theoretically analyzed. In addition, this article constructs an NSGP where the SGP is a degenerate case. Performance comparisons of the NSGP with the SGP and the NSGP-assisted EA (NSGP-MAEA) with the SGP-assisted EA (SGP-MAEA) are carried out on a set of benchmark problems and an antenna design problem. These comparison results demonstrate the competitiveness of the NSGP model.

Model based spectral feature analysis in reverberant acoustic fields

The spectral features of a reverberant acoustic field that can lead to improved source localization methods are investigated. Of particular interest are identification of deterministic and probabilistic features that characterize the environment and models of the acoustic transfer function that generate such features. For example, models that predict Schroeder’s invariant standard deviation in the pressure response at source-receiver distances where the direct sound approaches the reverberant field energy are proposed. Models examined include non-stationary Gaussian processes for the acoustic reverberant components and all-pass parametric models of the acoustic impulse response.

Stochastic variational inference for scalable non-stationary Gaussian process regression

A natural extension to standard Gaussian process (GP) regression is the use of non-stationary Gaussian processes, an approach where the parameters of the covariance kernel are allowed to vary in time or space. The non-stationary GP is a flexible model that relaxes the strong prior assumption of standard GP regression, that the covariance properties of the inferred functions are constant across the input space. Non-stationary GPs typically model varying covariance kernel parameters as further lower-level GPs, thereby enabling sampling-based inference. However, due to the high computational costs and inherently sequential nature of MCMC sampling, these methods do not scale to large datasets. Here we develop a variational inference approach to fitting non-stationary GPs that combines sparse GP regression methods with a trajectory segmentation technique. Our method is scalable to large datasets containing potentially millions of data points. We demonstrate the effectiveness of our approach on both synthetic and real world datasets.

Environmental sensor placement with convolutional Gaussian neural processes

Abstract Environmental sensors are crucial for monitoring weather conditions and the impacts of climate change. However, it is challenging to place sensors in a way that maximises the informativeness of their measurements, particularly in remote regions like Antarctica. Probabilistic machine learning models can suggest informative sensor placements by finding sites that maximally reduce prediction uncertainty. Gaussian process (GP) models are widely used for this purpose, but they struggle with capturing complex non-stationary behaviour and scaling to large datasets. This paper proposes using a convolutional Gaussian neural process (ConvGNP) to address these issues. A ConvGNP uses neural networks to parameterise a joint Gaussian distribution at arbitrary target locations, enabling flexibility and scalability. Using simulated surface air temperature anomaly over Antarctica as training data, the ConvGNP learns spatial and seasonal non-stationarities, outperforming a non-stationary GP baseline. In a simulated sensor placement experiment, the ConvGNP better predicts the performance boost obtained from new observations than GP baselines, leading to more informative sensor placements. We contrast our approach with physics-based sensor placement methods and propose future steps towards an operational sensor placement recommendation system. Our work could help to realise environmental digital twins that actively direct measurement sampling to improve the digital representation of reality.

Nonstationary Gaussian Process Discriminant Analysis With Variable Selection for High-Dimensional Functional Data

ABSTRACT High-dimensional classification and feature selection tasks are ubiquitous with the recent advancement in data acquisition technology. In several application areas such as biology, genomics, and proteomics, the data are often functional in their nature and exhibit a degree of roughness and nonstationarity. These structures pose additional challenges to commonly used methods that rely mainly on a two-stage approach performing variable selection and classification separately. We propose in this work a novel Gaussian process discriminant analysis (GPDA) that combines these steps in a unified framework. Our model is a two-layer nonstationary Gaussian process coupled with an Ising prior to identify differentially-distributed locations. Scalable inference is achieved via developing a variational scheme that exploits advances in the use of sparse inverse covariance matrices. We demonstrate the performance of our methodology on simulated datasets and two proteomics datasets: breast cancer and SARS-CoV-2. Our approach distinguishes itself by offering explainability as well as uncertainty quantification in addition to low computational cost, which are crucial to increase trust and social acceptance of data-driven tools. Supplementary materials for this article are available online.

Nonlinear wavelet-based estimation to spectral density for stationary non-Gaussian linear processes

Nonlinear wavelet-based estimators for spectral densities of non-Gaussian linear processes are considered. The convergence rates of mean integrated squared error (MISE) for those estimators over a large range of Besov function classes are derived, and it is shown that those rates are identical to minimax lower bounds in standard nonparametric regression model within a logarithmic term. Thus, those rates could be considered as nearly optimal. Therefore, the resulting wavelet-based estimators outperform traditional linear methods if the degree of smoothness of spectral densities varies considerably over the interval of interest, such as sharp spike, cusp, bump, etc., since linear estimators are not able to attain these rates. Unlike in classical nonparametric regression with Gaussian noise errors where thresholds are determined by normal distribution, we determine the thresholds based on a Bartlett type approximation of a quadratic form with dependent variables by its corresponding quadratic form with independent identically distributed (i.i.d.) random variables and Hanson-Wright inequality for quadratic forms in sub-gaussian random variables. The theory is illustrated with some numerical examples, and our simulation studies show that our proposed estimators are comparable to the current ones.

On the Gaussian and Non-Gaussian Characteristics of Nonstationary Seismic Ground Motions

In earthquake engineering, seismic ground motions are most often modelled as a nonstationary Gaussian process. A few studies indicated that seismic ground motions should be treated as a nonstationary non-Gaussian process, by showing that the kurtosis coefficient of the historical ground motion records is much greater than three. These findings and conclusions are queried in the present study, which analyzes a large number of historical ground motion records. Our results indicate that while the mixture marginal distribution of the acceleration of the records is non-Gaussian with a heavy distribution tail, the mixture marginal distribution of the standardized record, defined by the time-varying record to its standard deviation, is only mildly non-Gaussian. We point out that the mixture marginal distribution of a nonstationary Gaussian process may not be Gaussian. The implication of these observations in simulating records is explained. The sampled nonstationary Gaussian/non-Gaussian records are used to compare the responses of single-degree-of-freedom systems. The results indicate that the error introduced by adopting the Gaussian assumption is small, suggesting that the ground motions could be assumed to be a nonstationary Gaussian process with sufficient accuracy, especially if structures that are not very stiff are considered.

VisPro: a prognostic SqueezeNet and non-stationary Gaussian process approach for remaining useful life prediction with uncertainty quantification

VisPro: a prognostic SqueezeNet and non-stationary Gaussian process approach for remaining useful life prediction with uncertainty quantification

Detection and Analysis of Bionic Motion Pose of Single Leg and Hip Joint Based on Random Process.

Based on the spectral representation method of random function and combined with memoryless nonlinear translation theory, this paper analyzes the transformation relationship between potential Gaussian random process and non-Gaussian random process, and successfully generates a stationary non-Gaussian random process that conforms to the target non-Gaussian random process. For the non-stationary non-Gaussian random process simulation, on the basis of the stationary Gaussian random process, the intensity non-stationary uniform modulation model is used to modulate it, and combined with the nonlinear translation theory, the non-stationary non-Gaussian random process conforming to the target non-Gaussian random process is obtained. Aiming at the single-leg bouncing model based on the flexible rotary hip joint, the stability of its bouncing motion under passive motion is studied, and the influence of the flexible hip rotary joint on the motion stability is analyzed by comparing the single-leg bouncing motion characteristics of the free rotary hip joint. Based on the inverse dynamic control of the air phase, the fixed point distribution of the single-leg bounce of the flexible rotary hip joint was improved, and the function of the flexible rotary hip joint in the energy conversion of the bouncing motion was studied by establishing the energy consumption evaluation function. The kinematic performance verification, dynamic performance verification, dynamic parameter identification verification, and modal experiment simulation analysis were carried out for the built experimental platform, and the comparison and analysis with its theoretical model were carried out. The results show that the theoretical motion trajectory of the test mobile platform is basically consistent with the actual motion trajectory in the X and Y directions, and there is a small error in the Z-axis direction, and the error is within an acceptable range, indicating that the experimental platform system can be used to simulate the human hip joint. There is a large error between the theoretical value of the driving torque calculated by the theoretical value of the dynamic parameters and the measured value, and the dynamic theoretical model cannot accurately predict the driving torque. The predicted value of the driving torque calculated by using the identification value of the dynamic parameters is in good agreement with the measured torque, and its confidence is increased by 10–16%, indicating that the dynamic parameter identification method in this paper has a high degree of confidence.

A stochastic framework using Bayesian optimization algorithm to assess the network-level societal impacts of connected and autonomous vehicles

Connected and autonomous vehicle technologies are expected to alter transportation systems by enhancements in mobility, safety, and emission reduction. While many studies have investigated the impacts of connected vehicle (CV) and autonomous vehicle (AV) technologies on traffic congestion and emission at the facility level, little is known about these impacts at large scales. Furthermore, different effective parameters associated with these impacts, such as the extra vehicle miles traveled (VMT) induced by AVs, technology cost of these vehicles, and possible reductions in the value of time of AV users, are highly uncertain. The uncertainty of these parameters and its role in assessing the societal impacts of CVs and AVs are not fully explored in the literature. Therefore, this study aims to develop a stochastic framework and an optimization algorithm to find the optimum market shares of CVs and AVs in a mixed traffic environment, consisting of human-driven vehicles without connectivity (HDVs), CVs, and AVs, minimizing the system cost. Emission, travel time, and technology costs are considered as components of the system cost. Thus, the framework combines a traffic simulation tool that considers a mixed fleet of HDVs, CVs, and AVs with heterogeneous drivers (for HDVs and CVs) distributed spatially over the network, along with an emission estimation model, to measure network-wide travel time and emission costs. Many parameters, such as extra VMT produced by AVs, value of time reduction for AV users, and automation cost, are subject to a considerable degree of stochasticity, which is considered by assuming probabilistic distributions for these parameters. A Bayesian Optimization algorithm with heteroskedastic non-stationary Gaussian process model is presented to estimate the optimum market shares of CVs and AVs considering these uncertainties. The stochastic framework and the optimization algorithm are successfully applied to a large-scale network of Chicago. The impacts of different parameters on the optimum CV and AV market shares and the system cost are explored, providing practical insights for policymakers.

Simulating multivariate stationary non-Gaussian process based on wavenumber–frequency spectrum and unified Hermite polynomial model

It is of importance to model the input excitations considering the spatial variation and non-Gaussianity for the reliability evaluation of long span structures. The existing multivariate stationary non-Gaussian simulation methods are mainly based on the target marginal distribution and cross-spectrum matrix. Due to the Cholesky decomposition of cross-spectrum matrix, the computational cost of these methods increases rapidly with the increase of spatial points. Therefore, this paper develops a novel simulation method based on the wavenumber–frequency spectrum and unified Hermite polynomial model (UHPM). Firstly, UHPM is extended to the homogenous and stationary non-Gaussian field. Then, a complete transformation model from homogenous and stationary non-Gaussian auto-correlation function (ACF) into underlying Gaussian ACF with its applicable range is proposed. Furthermore, two types of incompatibility between the first four marginal moments and wavenumber–frequency spectrum are discussed, and the corresponding remedies are provided. To facilitate the calculation, the 2D-Fast Fourier Transform (FFT) technique is embedded in the Wiener–Khintchine transformation and spectral representation method (SRM). Finally, a unified simulation framework for multivariate stationary non-Gaussian process based on its relationship to the homogenous and stationary non-Gaussian field is presented. Two numerical examples, involving the simulations of non-Gaussian wind velocities and ground motion accelerations, are investigated to verify the capabilities of the proposed method.

Accurate Remaining Useful Life Prediction With Uncertainty Quantification: A Deep Learning and Nonstationary Gaussian Process Approach

Remaining useful life (RUL) refers to the expected remaining lifespan of a component or system. Accurate RUL prediction is critical for prognostic and health management and for maintenance planning. In this article, we address three prevalent challenges in data-driven RUL prediction, namely the handling of high-dimensional input features, the robustness to noise in sensor data and prognostic datasets, and the capturing of the time-dependency between system degradation and RUL prediction. We devise a highly accurate RUL prediction model with uncertainty quantification, which integrates and leverages the advantages of deep learning and nonstationary Gaussian process regression (DL–NSGPR). We examine and benchmark our model against other advanced data-driven RUL prediction models using the turbofan engine dataset from the NASA prognostic repository. Our computational experiments show that the DL-NSGPR predictions are highly accurate with root mean square error 1.7 to 6.2 times smaller than those of competing RUL models. Furthermore, the results demonstrate that RUL uncertainty bounds with the proposed DL-NSGPR are both valid and significantly tighter than other stochastic RUL prediction models. We unpack and discuss the reasons for this excellent performance of the DL-NSGPR.

PairGP: Gaussian process modeling of longitudinal data from paired multi-condition studies

High-throughput technologies produce gene expression time-series data that need fast and specialized algorithms to be processed. While current methods already deal with different aspects, such as the non-stationarity of the process and the temporal correlation, they often fail to take into account the pairing among replicates.We propose PairGP, a non-stationary Gaussian process method to compare gene expression time-series across several conditions that can account for paired longitudinal study designs and can identify groups of conditions that have different gene expression dynamics. We demonstrate the method on both simulated data and previously unpublished RNA sequencing (RNA-seq) time-series with five conditions. The results show the advantage of modeling the pairing effect to better identify groups of conditions with different dynamics. The pairing effect model displays good capabilities of selecting the most probable grouping of conditions even in the presence of a high number of conditions.The developed method is of general application and can be applied to any gene expression time series dataset. The model can identify common replicate effects among the samples coming from the same biological replicates and model those as separate components. Learning the pairing effect as a separate component, not only allows us to exclude it from the model to get better estimates of the condition effects, but also to improve the precision of the model selection process. The pairing effect that was accounted before as noise, is now identified as a separate component, resulting in more accurate and explanatory models of the data.