Multivariate Gaussian Process Research Articles

Efficient Method for Approximating the Joint Extreme Value Distribution of Multivariate Stationary Gaussian Processes

The joint extreme value distribution (JEVD) of multivariate random processes is important for evaluating the system reliability of a structure subjected to random vibrations. There are very limited analytical approaches for predicting the JEVD, and these approaches are only computationally viable for problems up to three dimensions. This paper presents an efficient approximate method, which is not limited to low-dimensional problems, for estimating the JEVD of multivariate stationary Gaussian processes. The proposed method modifies an existing method by using the Gauss-Legendre quadrature to evaluate the extreme value correlation coefficients under the bivariate Poisson assumption. Then, the JEVD is approximated using the Nataf transformation. The effectiveness of the proposed method is demonstrated via two numerical examples. The first example concerns the airgap problem of an offshore structure subjected to random waves, in which the extreme wave elevations are evaluated at six locations, and failure is defined as threshold exceedance for any location. The second example is a random vibration problem comprising a three degrees-of-freedom system. Previous studies focused on series system reliability; here three system reliabilities are examined, including series, parallel, and hybrid systems. The proposed method solves the problems efficiently, and it is found to provide an accurate prediction of the JEVD by comparison with Monte Carlo simulation.

Sample-path large deviations for stochastic evolutions driven by the square of a Gaussian process.

Recently, a number of physical models have emerged described by a random process with increments given by a quadratic form of a fast Gaussian process. We find that the rate function which describes sample-path large deviations for such a process can be computed from the large domain size asymptotic of a certain Fredholm determinant. The latter can be evaluated analytically using a theorem of Widom which generalizes the celebrated Szegő-Kac formula to the multidimensional case. This provides a large class of random dynamical systems with timescale separation for which an explicit sample-path large-deviation functional can be found. Inspired by problems in hydrodynamics and atmosphere dynamics, we construct a simple example with a single slow degree of freedom driven by the square of a fast multivariate Gaussian process and analyze its large-deviation functional using our general results. Even though the noiseless limit of this example has a single fixed point, the corresponding large-deviation effective potential has multiple fixed points. In other words, it is the addition of noise that leads to metastability. We use the explicit answers for the rate function to construct instanton trajectories connecting the metastable states.

Estimating human mobility in Holocene Western Eurasia with large-scale ancient genomic data

The recent increase in openly available ancient human DNA samples allows for large-scale meta-analysis applications. Trans-generational past human mobility is one of the key aspects that ancient genomics can contribute to since changes in genetic ancestry-unlike cultural changes seen in the archaeological record-necessarily reflect movements of people. Here, we present an algorithm for spatiotemporal mapping of genetic profiles, which allow for direct estimates of past human mobility from large ancient genomic datasets. The key idea of the method is to derive a spatial probability surface of genetic similarity for each individual in its respective past. This is achieved by first creating an interpolated ancestry field through space and time based on multivariate statistics and Gaussian process regression and then using this field to map the ancient individuals into space according to their genetic profile. We apply this algorithm to a dataset of 3138 aDNA samples with genome-wide data from Western Eurasia in the last 10,000 y. Finally, we condense this sample-wise record with a simple summary statistic into a diachronic measure of mobility for subregions in Western, Central, and Southern Europe. For regions and periods with sufficient data coverage, our similarity surfaces and mobility estimates show general concordance with previous results and provide a meta-perspective of genetic changes and human mobility.

Using Artificial Intelligence Approach for Investigating and Predicting Yield Stress of Cemented Paste Backfill

The technology known as cemented paste backfill (CPB) has gained considerable popularity worldwide. Yield stress (YS) is a significant factor considered in the assessment of CPB’s flowability or transportability. The minimal shear stress necessary to start the flow is known as Yield stress (YS), and it serves as an excellent measure of the strength of the particle-particle interaction. The traditional evaluation and measurement of YS performed by experimental tests are time-consuming and costly, which induces delays in construction projects. Moreover, the YS of CPB depends on numerous factors such as cement/tailing ratio, solid content and oxide content of tailing. Therefore, in order to simplify YS estimation and evaluation, the Artificial Intelligence (AI) approaches including eight Machine Learning techniques such as the Extreme Gradient Boosting algorithm, Gradient Boosting algorithm, Random Forest algorithm, Decision Trees, K-Nearest Neighbor, Support Vector Machine, Multivariate Adaptive Regression Splines and Gaussian Process are used to build the soft-computing model in predicting the YS of CPB. The performance of these models is evaluated by three metrics coefficient of determination (R2), Root Mean Square Error (RMSE) and Mean Absolute Error (MAE). The 3 best models were found to predict the Yield Stress of CPB (Gradient Boosting (GB), Extreme Gradient Boosting (XGB) and Random Forest (RF), respectively) with the 3 metrics of the three models, respectively, GB {R2 = 0.9811, RMSE = 0.1327 MPa, MAE = 0.0896 MPa}, XGB {R2 = 0.9034, RMSE = 0.3004 MPa, MAE = 0.1696 MPa} and RF {R2 = 0.8534, RMSE = 0.3700 MPa, MAE = 0.1786 MPa}, for the testing dataset. Based on the best performance model including GB, XG and RF, the other AI techniques such as SHapley Additive exPlanations (SHAP), Permutation Importance, and Individual Conditional Expectation (ICE) are also used for evaluating the factor effect on the YS of CPB. The results of this investigation can help the engineers to accelerate the mixed design of CPB.

Uncertainty modelling and dynamic risk assessment for long-sequence AIS trajectory based on multivariate Gaussian Process

A long-sequence multi-step prediction method based on multivariate Gaussian hypothesis and Gaussian process is proposed to model the uncertainty in the future ship path. This is a necessary step to predict the area where the ship is likely to be located at each future moment and to perform a dynamic risk assessment. Through data fusion, the uncertainty of the prediction is reduced, and more accurate support can be achieved for risk assessment. Firstly, from the current trajectory, the initial uncertainty intervals for the future trajectory are predicted based on the Gaussian process. Then, from the historical data, a reference trajectory set suitable for predicting the future path is generated based on a feature extracting process, named the reference trajectory prediction model in this paper, and the uncertainty intervals are also predicted. After that, the two parts are fused for a more accurate prediction to calculate the dynamic collision probability. The Gaussian process and a Laplacian Eigenmaps-Self-Organizing Maps model are adopted for fast batch processing. The experimental results demonstrate that the proposed model can combine the advantages of both and achieve a more accurate dynamic risk assessment.

Multivariate Gaussian processes: definitions, examples and applications

Gaussian processes occupy one of the leading places in modern statistics and probability theory due to their importance and a wealth of strong results. The common use of Gaussian processes is in connection with problems related to estimation, detection, and many statistical or machine learning models. In this paper, we propose a precise definition of multivariate Gaussian processes based on Gaussian measures on vector-valued function spaces, and provide an existence proof. In addition, several fundamental properties of multivariate Gaussian processes, such as stationarity and independence, are introduced. We further derive two special cases of multivariate Gaussian processes, including multivariate Gaussian white noise and multivariate Brownian motion, and present a brief introduction to multivariate Gaussian process regression as a useful statistical learning method for multi-output prediction problems.

Corrigendum: Exact Bayesian inference in spatiotemporal Cox processes driven by multivariate Gaussian processes

Corrigendum: Exact Bayesian inference in spatiotemporal Cox processes driven by multivariate Gaussian processes

Research on surface roughness prediction in turning Inconel 718 based on Gaussian process regression

Nickel-based alloy Inconel 718 is widely used in aircraft engine industry because of its good mechanical properties. Inconel 718 is a typical difficult-to-machine material and its price is relatively expensive. Therefore, accurate prediction of Inconel 718 machined surface roughness with small sample space can improve machining efficiency, optimize process parameters and reduce machining cost. In this paper, a method is proposed to characterize the influence of cutting parameters on roughness by stablishing the corresponding relationship between the proportional hyperparameters in the multivariate kernel function and the cutting speed, cutting deep, feed rate and the rake angle of the tool. A multi input single output (MISO) multivariate Gaussian process regression (GPR) surface roughness prediction model with cutting speed, cutting depth, feed rate and tool rake angle as input variables and surface roughness as output variables is established. The model can not only output the predicted value of surface roughness, but also give the reliability of the predicted value. Experimental results show that the proportional hyperparameter has an independent adjustment function, and the influence of the process parameters characterized by the proportional hyperparameter on the surface roughness is consistent with the experimental results. The experimental results show that the average relative error of MISO multivariate GPR surface roughness prediction model proposed in this paper is 1.5%, which can accurately predict the surface roughness in small sample space.

Estimating areal effects in typology: a case study of African phoneme inventories

Abstract In this paper, we combine several statistical techniques (multivariate probit models, Gaussian processes, and phylogenetic regression) into a new approach for exploring the areal patterns associated with the presence and absence of typological features. This model allows us to estimate and control for areal and genetic effects, the correlations between features, and their expected frequency of occurrence in the data. We use this model to explore the distribution of consonants in African languages, inferring several linguistic areas which have been proposed in the literature. Our results suggest that the combination of these methods is a promising new way of approaching language contact.

Robust optimization with uncertainty using a stochastic multivariate Gaussian process model

In robust parameter design (RPD), estimating the response surface model from the noisy experimental data is critical for parameter optimization. The multivariate Gaussian process (MGP) model has become a popular tool for response surface modelling. However, most existing MGP models may not effectively evaluate the uncertainty of noisy data. This article proposes a stochastic MGP for efficient emulation and robust optimization by considering uncertainty. Firstly, the hierarchical modelling technique is adopted to estimate the relationship model with noisy data. Secondly, the hyperparameters are estimated by the Gibbs technique, and the Bayesian averaging method is used to measure the parameter uncertainty. Finally, a novel optimization model integrated with the Bayesian averaging and quality loss function is developed to find the optimal solution. Two case studies are used to illustrate the effectiveness of the proposed method. The results show that the proposed approach obtains smaller quality loss than the existing ones.

Mixture of multivariate Gaussian processes for classification of irregularly sampled satellite image time-series

Mixture of multivariate Gaussian processes for classification of irregularly sampled satellite image time-series

Multivariate gaussian process incorporated predictive model for stream turbine power plant

Steam power turbine-based power plant approximately contributes 90% of the total electricity produced in the United States. Mainly steam turbine consists of multiple types of turbine, boiler, attemperator, reheater, etc. Power is produced through the steam with high pressure and temperature that is conducted by the turbines. The dynamics of the power plant are highly nonlinear considering all these elements in the model. We proposed to capture the dynamics of the power plant through Simulink modeling where data are generated, and a subsequent data-driven predictive modeling approach is used to detect the power generation from these turbines by the multivariate Gaussian process (MGP). The modeling approach is considered to predict the power generation from these turbines which can capture the cross-correlations between the turbines. Also, the sensitivity analysis of the input parameters is constructed for each turbine to find out the most important factors.

Investigation of Arcing Time Prediction for Secondary Arc Based on Statistical Theory and Multivariable Regression Algorithms

Research on the secondary arc has long been important in the development of ultrahigh-voltage (UHV) transmission technology. Predicting the arcing time of secondary arc is important for single-phase automatic reclosing of the circuit breaker to protect the power system. In response to this problem, this study uses a multifactor variance model to select factors that significantly influence the arcing time and verifies its validity through residual analysis. The results show that the parameters of the circuit, such as wind direction, recovery voltage, secondary current, and arc length, significantly influence the arcing time. The density-based clustering algorithm (DBSCAN) is then used to remove outliers form the data, and the influence of each factor on the arcing time is analyzed through scatterplots. Finally, linear multivariate regression and nonlinear Gaussian process regression are used to fit the results. The results show that the linear regression has a good imitative effect. The proposed method is accurate and provides novel means of predicting the arcing time.

Multiscale partial information decomposition of dynamic processes with short and long-range correlations: theory and application to cardiovascular control

Objective. In this work, an analytical framework for the multiscale analysis of multivariate Gaussian processes is presented, whereby the computation of Partial Information Decomposition measures is achieved accounting for the simultaneous presence of short-term dynamics and long-range correlations. Approach. We consider physiological time series mapping the activity of the cardiac, vascular and respiratory systems in the field of Network Physiology. In this context, the multiscale representation of transfer entropy within the network of interactions among Systolic arterial pressure (S), respiration (R) and heart period (H), as well as the decomposition into unique, redundant and synergistic contributions, is obtained using a Vector AutoRegressive Fractionally Integrated (VARFI) framework for Gaussian processes. This novel approach allows to quantify the directed information flow accounting for the simultaneous presence of short-term dynamics and long-range correlations among the analyzed processes. Additionally, it provides analytical expressions for the computation of the information measures, by exploiting the theory of state space models. The approach is first illustrated in simulated VARFI processes and then applied to H, S and R time series measured in healthy subjects monitored at rest and during mental and postural stress. Main Results. We demonstrate the ability of the VARFI modeling approach to account for the coexistence of short-term and long-range correlations in the study of multivariate processes. Physiologically, we show that postural stress induces larger redundant and synergistic effects from S and R to H at short time scales, while mental stress induces larger information transfer from S to H at longer time scales, thus evidencing the different nature of the two stressors. Significance. The proposed methodology allows to extract useful information about the dependence of the information transfer on the balance between short-term and long-range correlations in coupled dynamical systems, which cannot be observed using standard methods that do not consider long-range correlations.

Time-series forecasting using manifold learning, radial basis function interpolation, and geometric harmonics.

We address a three-tier numerical framework based on nonlinear manifold learning for the forecasting of high-dimensional time series, relaxing the "curse of dimensionality" related to the training phase of surrogate/machine learning models. At the first step, we embed the high-dimensional time series into a reduced low-dimensional space using nonlinear manifold learning (local linear embedding and parsimonious diffusion maps). Then, we construct reduced-order surrogate models on the manifold (here, for our illustrations, we used multivariate autoregressive and Gaussian process regression models) to forecast the embedded dynamics. Finally, we solve the pre-image problem, thus lifting the embedded time series back to the original high-dimensional space using radial basis function interpolation and geometric harmonics. The proposed numerical data-driven scheme can also be applied as a reduced-order model procedure for the numerical solution/propagation of the (transient) dynamics of partial differential equations (PDEs). We assess the performance of the proposed scheme via three different families of problems: (a) the forecasting of synthetic time series generated by three simplistic linear and weakly nonlinear stochastic models resembling electroencephalography signals, (b) the prediction/propagation of the solution profiles of a linear parabolic PDE and the Brusselator model (a set of two nonlinear parabolic PDEs), and (c) the forecasting of a real-world data set containing daily time series of ten key foreign exchange rates spanning the time period 3 September 2001-29 October 2020.

Study of fluid–thermal–structural interaction in high-temperature high-speed flow using multi-fidelity multi-variate surrogates

This study investigates the impact of the high-temperature effect, especially the real gas effect and chemical reactions, on hypersonic aerothermodynamic solutions of double cone and double wedge configurations, as well as the fluid–thermal–structural interaction of a double wedge configuration in hypersonic flow. First, a high-temperature computational fluid dynamics (CFD) code was benchmarked and correlated with experimental results, emphasizing the impact of high-temperature effects as well as turbulence modeling on heat flux prediction. Subsequently, the multi-fidelity multi-variate Gaussian process regression (M2GPR ) method for problems with high-dimensional outputs was developed to create an aerothermal surrogate model. The model achieves a balance between model accuracy and computational cost of sample generation, using the combination of a few high-fidelity samples and many low-fidelity samples. The numerical examples show that, using the M2GPR formulation, the required number of high-fidelity samples may be reduced by over 80% while maintaining an accuracy comparable to the high-fidelity CFD solvers. In addition, a geodesic-distance-based metric is developed to inform the choice of high-dimensional datasets of different fidelities for the M2GPR surrogate with improved accuracy. Finally, the aerothermal surrogate was applied to study the impact of the high-temperature effect on the aerothermoelastic response of a hypersonic skin panel, emphasizing the necessity of the accurate characterization of the localized heat flux for reasonable assessment of the response of a compliant structure in high-speed high-temperature flowfield.

Comprehensively testing linearity hypothesis using the smooth transition autoregressive model

This article examines the null limit distribution of the quasi-likelihood ratio (QLR) statistic for testing linearity condition against the smooth transition autoregressive (STAR) model. We explicitly show that the QLR test statistic weakly converges to a functional of a multivariate Gaussian process under the null of linearity, which is done by resolving the issue of identification problem arises in two different ways under the null. In contrast with the Lagrange multiplier test that is widely employed for testing the linearity condition, the proposed QLR statistic has an omnibus power, and thus, it complements the existing testing procedure. We show the empirical relevance of our test by testing the neglected nonlinearity of the US fiscal multipliers and growth rates of US unemployment. These empirical examples demonstrate that the QLR test is useful for detecting the nonlinear structure among economic variables.

Comprehensive uncertainty estimation of the timing of Greenland warmings in the Greenland ice core records

Abstract. Paleoclimate proxy records have non-negligible uncertainties that arise from both the proxy measurement and the dating processes. Knowledge of the dating uncertainties is important for a rigorous propagation to further analyses, for example, for identification and dating of stadial–interstadial transitions in Greenland ice core records during glacial intervals, for comparing the variability in different proxy archives, and for model-data comparisons in general. In this study we develop a statistical framework to quantify and propagate dating uncertainties in layer counted proxy archives using the example of the Greenland Ice Core Chronology 2005 (GICC05). We express the number of layers per depth interval as the sum of a structured component that represents both underlying physical processes and biases in layer counting, described by a regression model, and a noise component that represents the fluctuations of the underlying physical processes, as well as unbiased counting errors. The joint dating uncertainties for all depths can then be described by a multivariate Gaussian process from which the chronology (such as the GICC05) can be sampled. We show how the effect of a potential counting bias can be incorporated in our framework. Furthermore we present refined estimates of the occurrence times of Dansgaard–Oeschger events evidenced in Greenland ice cores together with a complete uncertainty quantification of these timings.

Improving exoplanet detection power: Multivariate Gaussian process models for stellar activity

The radial velocity method is one of the most successful techniques for detecting exoplanets. It works by detecting the velocity of a host star, induced by the gravitational effect of an orbiting planet, specifically, the velocity along our line of sight which is called the radial velocity of the star. Low-mass planets typically cause their host star to move with radial velocities of 1 m/s or less. By analyzing a time series of stellar spectra from a host star, modern astronomical instruments can, in theory, detect such planets. However, in practice, intrinsic stellar variability (e.g., star spots, convective motion, pulsations) affects the spectra and often mimics a radial velocity signal. This signal contamination makes it difficult to reliably detect low-mass planets. A principled approach to recovering planet radial velocity signals in the presence of stellar activity was proposed by Rajpaul et al. (Mon. Not. R. Astron. Soc. 452 (2015) 2269–2291). It uses a multivariate Gaussian process model to jointly capture time series of the apparent radial velocity and multiple indicators of stellar activity. We build on this work in two ways: (i) we propose using dimension reduction techniques to construct new high-information stellar activity indicators; and (ii) we extend the Rajpaul et al. (Mon. Not. R. Astron. Soc. 452 (2015) 2269–2291) model to a larger class of models and use a power-based model comparison procedure to select the best model. Despite significant interest in exoplanets, previous efforts have not performed large-scale stellar activity model selection or attempted to evaluate models based on planet detection power. In the case of main sequence G2V stars, we find that our method substantially improves planet detection power, compared to previous state-of-the-art approaches.

Emulating Spatial and Temporal Outputs From Fuel Cell and Battery Models: A Comparison of Deep Learning and Gaussian Process Models

Abstract Neural network models have a long history in fuel cell and battery modeling. With the recent advent of deep learning, there is potential for further improvements in these models. Conversely, deep learning is primarily designed for image detection and classification using large data sets and its performance on typical regression tasks in fuel cell and battery modeling remains largely unexplored. In this article, we present a new method for applying deep learning to general vector outputs from battery and fuel cell models and investigate the use of different deep learning architectures. We compare these methods to equivalent Gaussian process (GP) models on a range of regression tasks. We further provide the first rigorous error and asymptotic analysis of the multivariate GP model. For scalar outputs, deep networks are found to be less accurate on small data sets, but for large data sets, convolutional and recurrent networks are able to marginally exceed the accuracy of GP models.