Independent Gaussian Processes Research Articles

Data-driven model selections of second-order particle dynamics via integrating Gaussian processes with low-dimensional interacting structures

In this paper, we focus on the data-driven discovery of a general second-order particle-based model that contains many state-of-the-art models for modeling the aggregation and collective behavior of interacting agents of similar size and body type. This model takes the form of a high-dimensional system of ordinary differential equations parameterized by two interaction kernels that appraise the alignment of positions and velocities. We propose a Gaussian Process-based approach to this problem, where the unknown model parameters are marginalized by using two independent Gaussian Process (GP) priors on latent interaction kernels constrained to dynamics and observational data. This results in a nonparametric model for interacting dynamical systems that accounts for uncertainty quantification. We also show that our estimators can be efficiently derived through the use of preconditioned techniques, ensuring scalability. Moreover, we perform a theoretical analysis to interpret the methodology and investigate the conditions under which the kernels can be recovered. We demonstrate the effectiveness of the proposed approach on various prototype systems, including the selection of the order of the systems and the types of interactions. In particular, we present applications to modeling two real-world fish motion datasets that display flocking and milling patterns up to 248 dimensions. Despite the use of small datasets, the GP-based approach learns an effective representation of the nonlinear dynamics in these spaces and outperforms competitor methods.

Exposure time and point cloud quality prediction for active 3D imaging sensors using Gaussian process regression

Setting an optimal image exposure is crucial for acquiring dense point clouds using 3D active optical sensor systems such as structured light sensors [structured light sensors (SLSs)] and active stereo sensors. One of the most common and seamless ways to optimize the image brightness of an image exposure is to adjust the camera’s exposure time. However, optimizing the image exposure alone is ineffective for acquiring surfaces of large-scale objects with a complex topology if a spatial understanding of the scene is neglected. Hence, the present paper proposes a data-driven approach using two Gaussian processes [Gaussian processes (GPs)] regression models to select a proper exposure time considering the nonlinear correlations between image exposure and the scene spatial relationships. To model these correlations, our study introduces first the generic synthesization of seven inputs and two target variables. Then, based on these inputs, two independent GPs are designed: one for predicting the measurement quality and one for estimating the exposure time. The performance and generalizability of both models are thoroughly evaluated using an SLS and an active stereo sensor. The evaluation demonstrated that the point cloud quality models adequately matched observations with an R2 exceeding 90%. Specifically, the models predicted point cloud quality with an root mean square error (RMSE) of 10%. Additionally, the assessment of the performance of the exposure time models showed a model fit with an R2 above 97%. The exposure time prediction accuracy, as evidenced by the RMSE values, was within 10% of the corresponding exposure time range for each sensor. The present research shows the potential and effectiveness to completely automate the assessment of a point cloud quality and the selection of exposure times with the help of data-driven models.

Multi-objective Bayesian alloy design using multi-task Gaussian processes

In design applications, correlations among material properties (such as the tendency for stronger materials to be less ductile) are often neglected. This approach is echoed in multi-objective optimization techniques which treat each performance characteristic as an independent objective, aiming to optimize scalar functions and find optimal Pareto fronts. However, this overlooks the statistical relationships between performance characteristics inherent in a material system. To address this, we propose the use of Bayesian optimization, a highly efficient black-box optimization algorithm known for constructing Gaussian processes (GPs) – uncorrelated surrogates - to model objective functions. Rather than evaluating multiple GPs for each objective function separately, we argue for a shift towards jointly modeling these objective functions, considering their statistical correlations. This integrated approach utilizes naturally occurring relationships among material properties, providing additional information to enhance the performance of the design framework. This requires the replacement of multiple independent GPs with a single multi-task GP, employing a correlation matrix to construct a multi-task kernel function, wherein each task corresponds to a single objective function. We anticipate this refined methodology will better leverage material correlations, improving design optimization results.

Joint Modeling of Radial Velocities and Photometry with a Gaussian Process Framework

Developments in the stability of modern spectrographs have led to extremely precise instrumental radial velocity (RV) measurements. For most stars, the detection limit of planetary companions with these instruments is expected to be dominated by astrophysical noise sources such as starspots. Correlated signals caused by rotationally modulated starspots can obscure or mimic the Doppler shifts induced by even the closest, most massive planets. This is especially true for young, magnetically active stars where stellar activity can cause fluctuation amplitudes of ≳0.1 mag in brightness and ≳100 m s−1 in RV semiamplitudes. Techniques that can mitigate these effects and increase our sensitivity to young planets are critical to improving our understanding of the evolution of planetary systems. Gaussian processes (GPs) have been successfully employed to model and constrain activity signals in individual cases. However, a principled approach of this technique, specifically for the joint modeling of photometry and RVs, has not yet been developed. In this work, we present a GP framework to simultaneously model stellar activity signals in photometry and RVs that can be used to investigate the relationship between both time series. Our method, inspired by the FF framework of Aigrain et al., models spot-driven activity signals as the linear combinations of two independent latent GPs and their time derivatives. We also simulate time series affected by starspots by extending the starry software to incorporate time evolution of stellar features. Using these synthetic data sets, we show that our method can predict spot-driven RV variations with greater accuracy than other GP approaches.

Gaussian process hydrodynamics

We present a Gaussian process (GP) approach, called Gaussian process hydrodynamics (GPH) for approximating the solution to the Euler and Navier-Stokes (NS) equations. Similar to smoothed particle hydrodynamics (SPH), GPH is a Lagrangian particle-based approach that involves the tracking of a finite number of particles transported by a flow. However, these particles do not represent mollified particles of matter but carry discrete/partial information about the continuous flow. Closure is achieved by placing a divergence-free GP prior ξ on the velocity field and conditioning it on the vorticity at the particle locations. Known physics (e.g., the Richardson cascade and velocity increment power laws) is incorporated into the GP prior by using physics-informed additive kernels. This is equivalent to expressing ξ as a sum of independent GPs ξl, which we call modes, acting at different scales (each mode ξl self-activates to represent the formation of eddies at the corresponding scales). This approach enables a quantitative analysis of the Richardson cascade through the analysis of the activation of these modes, and enables us to analyze coarse-grain turbulence statistically rather than deterministically. Because GPH is formulated by using the vorticity equations, it does not require solving a pressure equation. By enforcing incompressibility and fluid-structure boundary conditions through the selection of a kernel, GPH requires significantly fewer particles than SPH. Because GPH has a natural probabilistic interpretation, the numerical results come with uncertainty estimates, enabling their incorporation into an uncertainty quantification (UQ) pipeline and adding/removing particles (quanta of information) in an adapted manner. The proposed approach is suitable for analysis because it inherits the complexity of state-of-the-art solvers for dense kernel matrices and results in a natural definition of turbulence as information loss. Numerical experiments support the importance of selecting physics-informed kernels and illustrate the major impact of such kernels on the accuracy and stability. Because the proposed approach uses a Bayesian interpretation, it naturally enables data assimilation and predictions and estimations by mixing simulation data and experimental data.

Simultaneous ruin probability for multivariate Gaussian risk model

Let Z(t)=(Z1(t),…,Zd(t))⊤,t∈R where Zi(t),t∈R, i=1,…,d are mutually independent centered Gaussian processes with continuous sample paths a.s. and stationary increments. For X(t)=AZ(t),t∈R, where A is a nonsingular d×d real-valued matrix, u,c∈Rd and T>0 we derive tight bounds for P∃t∈[0,T]:∩i=1d{Xi(t)−cit>ui}and find exact asymptotics as (u1,…,ud)⊤=(ua1,…,uad)⊤ for any (a1,…,ad)⊤∈Rd∖(−∞,0]d and u→∞.

АНАЛІЗ ЗАСТОСОВНОСТІ МЕТОДУ ДВОЧАСТОТНОЇ ІНТЕРФЕРОМЕТРІЇ ДЛЯ ВИМІРЮВАННЯ КУТА МІСЦЯ ЦІЛЕЙ У ДВОКООРДИНАТНИХ РЛС

Subject and Purpose. The study deals with the dual-frequency radio interferometry technique, which is based on the employment of two fairly close frequencies with the aim to remove ambiguity of the radar target elevation estimation using 2D-radar and eliminate 2 pm-uncertainty of the signal phase difference measurement. Analysis of random noise action on the accuracy of the elevation angle estimation by the dual-frequency radio interferometry and assessment of practical applicability of the method make up the purpose of the paper. Methods and Methodology. The noise action on the elevation angle measurement accuracy is examined through a series of an- alytical calculations with the use of statistical analysis methods. The noise in each receiving channel is modeled in terms of additive, statistically independent stationary Gaussian processes with zero mean values and equal variances. The calculation results are checked via computer simulations with statistics estimations for 106 random noise realizations. Results. A correct condition has been developed for the sector width where the target elevation angle is unambiguously estimated depending on the space separation of the antennas (baselines) and the frequency ratio. Expressions for elevation angle estimation errors have been obtained, showing that the error is mainly contributed by the faults in the determination of the ambiguity interval number. A probability of the correct determination of the ambiguity interval number has been derived depending on the signal- to-noise ratio and the frequency difference, indicating that almost one hundred per cent probability of the correct determination of the ambiguity interval number is only achieved when the signal-to-noise ratio exceeds 30 dB. A comparative analysis has been performed between the methods of dual-frequency interferometry and conventional phase-difference direction finding in the case of close X-band frequencies and the same sectors of survey. Conclusions. The dual-frequency radio interferometry technique with close frequencies has been shown to outperform the stand- ard phase-difference direction-finding method only when the signal-to-noise ratio is sufficiently high (over 30 dB). In principle, the accuracy of the technique seems possible to improve by taking significantly different frequencies selected with regard to the scale negotiation condition. However, it should be mentioned that the implementation of the relevant algorithm in practice is much more complicated than the conventional scheme with a single frequency and several antenna baselines.

Multi-Patch Epidemic Models with General Exposed and Infectious Periods

We study multi-patch epidemic models where individuals may migrate from one patch to another in either of the susceptible, exposed/latent, infectious and recovered states. We assume that infections occur both locally with a rate that depends on the patch as well as “from distance” from all the other patches. The migration processes among the patches in either of the four states are assumed to be Markovian, and independent of the exposed and infectious periods. These periods have general distributions, and are not affected by the possible migrations of the individuals. The infection “from distance” aspect introduces a new formulation of the infection process, which, together with the migration processes, brings technical challenges in proving the functional limit theorems. Generalizing the methods in Pang and Pardoux [Ann. Appl. Probab. 32 (2022) 1615–1665], we establish a functional law of large number (FLLN) and a function central limit theorem (FCLT) for the susceptible, exposed/latent, infectious and recovered processes. In the FLLN, the limit is determined by a set of Volterra integral equations. In the special case of deterministic exposed and infectious periods, the limit becomes a system of ODEs with delays. In the FCLT, the limit is given by a set of stochastic Volterra integral equations driven by a sum of independent Brownian motions and continuous Gaussian processes with an explicit covariance structure.

Epidemics from the Eye of the Pathogen

While a common trend in disease modeling is to develop models of increasing complexity, it was recently pointed out that outbreaks appear remarkably simple when viewed in the incidence vs. cumulative cases (ICC) plane. This article details the theory behind this phenomenon by analyzing the stochastic SIR (Susceptible, Infected, Recovered) model in the cumulative cases domain. We prove that the Markov chain associated with this model reduces, in the ICC plane, to a pure birth chain for the cumulative number of cases, whose limit leads to an independent increments Gaussian process that fluctuates about a deterministic ICC curve. We calculate the associated variance and quantify the additional variability due to estimating incidence over a finite period of time. We also illustrate the universality brought forth by the ICC concept on real-world data for Influenza A and for the COVID-19 outbreak in Arizona.

A Gaussian version of Littlewood’s theorem for random power series

We prove a Littlewood-type theorem for random analytic functions associated with not necessarily independent Gaussian processes. We show that if we randomize a function in the Hardy space H 2 ( D ) H^2(\mathbb {D}) by a Gaussian process whose covariance matrix K K induces a bounded operator on l 2 l^2 , then the resulting random function is almost surely in H p ( D ) H^p(\mathbb {D}) for any p > 0 p>0 . The case K = I d K=\mathrm {Id} , the identity operator, recovers Littlewood’s theorem. A new ingredient in our proof is to recast the membership problem as the boundedness of an operator. This reformulation enables us to use tools in functional analysis and is applicable to other situations.

Scalable multi-task Gaussian processes with neural embedding of coregionalization

Multi-task regression attempts to exploit the task similarity in order to achieve knowledge transfer across related tasks for improving the quality of prediction and alleviating the demand of big data. The application of Gaussian process (GP) in this scenario yields the non-parametric yet informative Bayesian multi-task regression paradigm. Multi-task GP (MTGP) provides not only the prediction mean but also the associated prediction variance to quantify uncertainty, thus gaining popularity in various scenarios. The linear model of coregionalization (LMC) is a well-known MTGP paradigm which exploits the dependency of tasks through linear combination of several independent and diverse GPs. The LMC however suffers from high model complexity and limited model capability when handling complicated multi-task cases. To this end, we develop the neural embedding of coregionalization that transforms the latent GPs into a high-dimensional latent space to induce rich yet diverse behaviors. Furthermore, we use advanced variational inference as well as sparse approximation to devise a tight and compact evidence lower bound (ELBO) for higher quality of scalable model inference. Extensive numerical experiments have been conducted to verify the higher prediction quality and better generalization of our model, named NSVLMC, on various real-world multi-task datasets and the cross-fluid modeling of unsteady fluidized bed.

Path Loss Model Based on Machine Learning Using Multi-Dimensional Gaussian Process Regression

For beyond the fifth-generation (5G) and future wireless communications, spatial consistency that represents the correlation between propagation channel characteristics in close proximity has become one of the major issues in channel modeling to describe channels more realistically in emerging scenarios, such as device-to-device (D2D). In this study, we propose a novel path loss model based on multi-dimensional Gaussian process regression (GPR) that provides spatial consistency to channels in propagation environment by predicting local shadow fading while fitting large-scale path loss from measured data. The proposed model has a special structure consisting of a radial mean function and local shadow fading term, which are modeled by independent Gaussian Processes. In contrast to the log-distance path loss model and other regression-based models, the proposed model trains two functions simultaneously; thus, predicts path loss well by capturing both global tendency and local correlation. Moreover, because the proposed model is based on GPR, it provides uncertainty of the predicted path loss. We validated the performance of the proposed model in terms of prediction accuracy with the measurement datasets from two different indoor environments. Our experiments showed that the proposed model predicts better than the log-distance path loss model, especially when spatial correlation becomes significant. The proposed model can be used to simulate path loss in a general environment after training the measurement data.

Spin distributions for generic spherical spin glasses

This paper investigates spin distributions for a generic spherical p-spin model; we give a representation of spin distributions in terms of a stochastic process. In order to do this, we find a series of invariance principles analogous to the cavity equations used for the hypercube. The construction of these equations is a multistep process; we first change coordinates to write the sphere as a product space SN−R2×[−C,C] for some cutoff parameters R and C. We then construct an auxiliary Hamiltonian that writes the effect of the final spin as an independent Gaussian process, e.g., HN≈HN−1+YN+ZN with YN and ZN decoupled from HN−1. The error rate in this approximation decays as O(C4R2). The final step is to justify the double limit limC→∞limR→∞ to derive exact results; this double limit can be interpreted as a novel renormalization procedure.

A functional limit theorem for nested Karlin's occupancy scheme generated by discrete Weibull-like distributions

Let (pk)k∈N be a discrete probability distribution for which the counting function x↦#{k∈N:pk≥1/x} belongs to the de Haan class Π. Consider a deterministic weighted branching process generated by (pk)k∈N. A nested Karlin's occupancy scheme is the sequence of Karlin balls-in-boxes schemes in which boxes of the jth level, j=1,2,… are identified with the jth generation individuals and the hitting probabilities of boxes are identified with the corresponding weights. The collection of balls is the same for all generations, and each ball starts at the root and moves along the tree of the deterministic weighted branching process according to the following rule: transition from a mother box to a daughter box occurs with probability given by the ratio of the daughter and mother weights. Assuming there are n balls, denote by Kn(j) the number of occupied (ever hit) boxes in the jth level. For each j∈N, we prove a functional limit theorem for the vector-valued process (K⌊eT+u⌋(1),…,K⌊eT+u⌋(j))u∈R, properly normalized and centered, as T→∞. The limit is a vector-valued process whose components are independent stationary Gaussian processes. An integral representation of the limit process is obtained.

A high-efficiency simulation method of wind field and its application on transmission line

Generally, the fluctuating wind is simplified as several independent one-dimensional multivariate stationary Gaussian processes in simulating a natural wind field. The correlation in the lateral, longitudinal and vertical directions should all be considered in the simulation of longitudinal wind field for the large-span spatial structures. In fact, this type of structure has lots of simulation points. The calculation amount of wind field simulation by the harmonic superposition method depends on the scale of cross-spectral density matrix, which is directly related to the number of simulated points, leading to a low efficiency when generating the time-varying wind speed. This paper innovatively proposes a high-efficiency simulation method for the longitudinal wind field based on Taylor's hypothesis. Subsequently, the effectiveness of the proposed wind field method was verified by the numerical simulation. Finally, the dynamic responses of a transmission tower-line system under the wind loadings generated with the new method and traditional method are calculated and compared. The percentages difference of the mean and maximum axial force at the main tower members are less than 0.02% and 1%, respectively, indicating the effectiveness of the proposed time delay method. The results also show that the proposed simulation method of wind field can not only ensure the simulation accuracy, but also significantly improve the efficiency of wind speed generation, which is suitable for the wind load simulation of large-span spatial structures.

Bayesian Pure-Tone Audiometry Through Active Learning Under Informed Priors.

Pure-tone audiometry—the process of estimating a person's hearing threshold from “audible” and “inaudible” responses to tones of varying frequency and intensity—is the basis for diagnosing and quantifying hearing loss. By taking a probabilistic modeling approach, both optimal tone selection (in terms of expected information gain) and hearing threshold estimation can be derived through Bayesian inference methods. The performance of probabilistic model-based audiometry methods is directly linked to the quality of the underlying model. In recent years, Gaussian process (GP) models have been shown to provide good results in this context. We present methods to improve the efficiency of GP-based audiometry procedures by improving the underlying model. Instead of a single GP, we propose to use a GP mixture model that can be conditioned on side-information about the subject. The underlying idea is that one can typically distinguish between different types of hearing thresholds, enabling a mixture model to better capture the statistical properties of hearing thresholds among a population. Instead of modeling all hearing thresholds by a single GP, a mixture model allows specific types of hearing thresholds to be modeled by independent GP models. Moreover, the mixing coefficients can be conditioned on side-information such as age and gender, capturing the correlations between age, gender, and hearing threshold. We show how a GP mixture model can be optimized for a specific target population by learning the parameters from a data set containing annotated audiograms. We also derive an optimal tone selection method based on greedy information gain maximization, as well as hearing threshold estimation through Bayesian inference. The proposed models are fitted to a data set containing roughly 176 thousand annotated audiograms collected in the Nordic countries. We compare the predictive accuracies of optimized mixture models of varying sizes with that of an optimized single-GP model. The usefulness of the optimized models is tested in audiometry simulations. Simulation results indicate that an optimized GP mixture model can significantly outperform an optimized single-GP model in terms of predictive accuracy, and leads to significant increases the efficiency of the resulting Bayesian audiometry procedure.

Spatial Shrinkage Via the Product Independent Gaussian Process Prior

We study the problem of sparse signal detection on a spatial domain. We propose a novel approach to model continuous signals that are sparse and piecewise-smooth as the product of independent Gaussian (PING) processes with a smooth covariance kernel. The smoothness of the PING process is ensured by the smoothness of the covariance kernels of the Gaussian components in the product, and sparsity is controlled by the number of components. The bivariate kurtosis of the PING process implies that more components in the product results in the thicker tail and sharper peak at zero. We develop an efficient computation algorithm based on spectral methods. The simulation results demonstrate superior estimation using the PING prior over Gaussian process prior for different image regressions. We apply our method to a longitudinal magnetic resonance imaging dataset to detect the regions that are affected by multiple sclerosis computation in this domain. Supplementary materials for this article are available online.

Derivatives of local times for some Gaussian fields II

Given a (2,d)-Gaussian field Z={Z(t,s)=XtH1−X˜sH2,s,t≥0},where XH1 and X˜H2 are independent d-dimensional centered Gaussian processes satisfying certain properties, we will give a necessary condition for existence of derivatives of the local time of Z at x∈Rd, this condition is also sufficient when XH1 and X˜H2 satisfy the local nondeterminism property.

Prediction of non-stationary response functions using a Bayesian composite Gaussian process

The modeling and prediction of functions that can exhibit non-stationarity characteristics is important in many applications; for example, this is often the case for simulator output. One approach to predict a function with unknown stationarity properties is to model it as a draw from a flexible stochastic process that can produce stationary or non-stationary realizations. One such model is the composite Gaussian process (CGP) which expresses the large-scale (global) trends of the output and the small-scale (local) adjustments to the global trend as independent Gaussian processes; an extension of the CGP model can produce realizations with non-constant variance by allowing the variance of the local process to vary over the input space. A new, Bayesian extension of a global-trend plus local-trend model is proposed that also allows measurement errors. In contrast to the original CGP model, the new Bayesian CGP model introduces a weight function to allow the total process variability to be apportioned between the large- and small-scale processes. The proposed prior distributions ensure that the fitted global mean is smoother than the local deviations, a feature built into the CGP model. The log of the process variance for the Bayesian CGP is modeled as a Gaussian process to provide a flexible mechanism for handling variance functions that vary across the input space. A Markov chain Monte Carlo algorithm is proposed that provides posterior estimates of the parameters for the Bayesian CGP. It also yields predictions of the output and quantifies uncertainty about the predictions. The method is illustrated using both analytic and real-data examples.

TOC content prediction based on a combined Gaussian process regression model

Continuous total organic carbon (TOC) content estimation plays a crucial role in source rock reservoir evaluation. However, conventional prediction methods (e.g., empirical formulas or the δlogR method) have low prediction accuracy. Although machine learning algorithms possess a better prediction accuracy, they have also reached a bottleneck in this regard. Recently, ensemble learning and combined model prediction have achieved excellent results in many fields. In this study, a combined model was used for building a more accurate prediction model. Based on the theory that different formations correspond to different logging responses, we hypothesized that a Gaussian mixture model (GMM) classification of the logging data would be able to represent different geological conditions. Hence, we constructed independent Gaussian process regression (GPR) model based on the classified data; in particular, the posterior probability given by the GMM was used as a weight coefficient to combine the prediction results of all the independent GPR models. In addition, a confidence interval was obtained through the GPR to quantify the uncertainty of the prediction results. Finally, an error analysis showed that the combined-GPR model performed better than the δlogR method and other machine learning algorithms. In conclusion, the combined prediction model proposed in this study is reliable and effective, and it provides a new solution for quantitative prediction problems in corresponding fields.