Non-stationary Covariance Functions Research Articles

Locally Anisotropic Nonstationary Covariance Functions on the Sphere

Locally Anisotropic Nonstationary Covariance Functions on the Sphere

Deep Kernel Learning-Based Bayesian Optimization with Adaptive Kernel Functions

Bayesian optimization (BO) is a widely used data-driven method for the global optimization of black-box objective functions with noise. A key component of BO is surrogate model that is a probabilistic model used for approximating black-box objective functions. However, classical BO methods often utilize Gaussian processes with stationary covariance functions as surrogate models, which is difficult to accurately model complex systems with rapidly oscillating target functions, and therefore deteriorates the performance of BO. To this end, a Bayesian optimization approach based on deep kernel learning (DKL) is investigated for the optimization of complex objective functions with large oscillations, in which a Gaussian process based on deep kernel learning is utilized to capture the characteristics of data with non-stationary and hierarchical covariance functions via deep neural networks. The optimization performance of the DKL-based BO approach with different network structures is quantified and compared with a state-of-the-art BO method based on Gaussian process under the scenarios of different noise levels.

Parametric nonstationary covariance functions on spheres

Gaussian Processes are powerful tools for modelling spatial data. In this context, a significant amount of modelling focus is placed on specifying the covariance function, which is required to be symmetric and positive definite. Covariance functions have classically been defined and used in Euclidean space. However, as data collected from the globe becomes more prevalent, accounting for Earth's geometry becomes increasingly important. Using Euclidean distance can be suboptimal for these data. We survey the literature for recent developments related to construction of nonstationary covariance functions on spheres, which historically has been a challenging area. We present contributions in this effort by providing three general forms for families of parametric nonstationary covariance functions.

Change-point detection in hierarchical circadian models

This paper addresses the problem of change-point detection in sequences of high-dimensional and heterogeneous observations, which also possess a periodic temporal structure. Due to the dimensionality problem, when the time between change points is of the order of the dimension of the model parameters, drifts in the underlying distribution can be misidentified as changes. To overcome this limitation, we assume that the observations lie in a lower-dimensional manifold that admits a latent variable representation. In particular, we propose a hierarchical model that is computationally feasible, widely applicable to heterogeneous data and robust to missing instances. Additionally, the observations’ periodic dependencies are captured by non-stationary periodic covariance functions. The proposed technique is particularly well suited to (and motivated by) the problem of detecting changes in human behavior using smartphones and its application to relapse detection in psychiatric patients. Finally, we validate the technique on synthetic examples and we demonstrate its utility in the detection of behavioral changes using real data acquired by smartphones.

SMOOTH DENSITY SPATIAL QUANTILE REGRESSION.

We derive the properties and demonstrate the desirability of a model-based method for estimating the spatially-varying effects of covariates on the quantile function. By modeling the quantile function as a combination of I-spline basis functions and Pareto tail distributions, we allow for flexible parametric modeling of the extremes while preserving non-parametric flexibility in the center of the distribution. We further establish that the model guarantees the desired degree of differentiability in the density function and enables the estimation of non-stationary covariance functions dependent on the predictors. We demonstrate through a simulation study that the proposed method produces more efficient estimates of the effects of predictors than other methods, particularly in distributions with heavy tails. To illustrate the utility of the model we apply it to measurements of benzene collected around an oil refinery to determine the effect of an emission source within the refinery on the distribution of the fence line measurements.

Reduction problems and deformation approaches to nonstationary covariance functions over spheres

The paper considers reduction problems and deformation approaches for nonstationary covariance functions on the $(d-1)$-dimensional spheres, $\mathbb{S}^{d-1}$, embedded in the $d$-dimensional Euclidean space. Given a covariance function $C$ on $\mathbb{S}^{d-1}$, we chase a pair $(R,\Psi)$, for a function $R:[-1,+1]\to \mathbb{R}$ and a smooth bijection $\Psi$, such that $C$ can be reduced to a geodesically isotropic one: $C(\mathbf{x},\mathbf{y})=R(\langle \Psi (\mathbf{x}),\Psi (\mathbf{y})\rangle )$, with $\langle \cdot ,\cdot \rangle $ denoting the dot product. The problem finds motivation in recent statistical literature devoted to the analysis of global phenomena, defined typically over the sphere of $\mathbb{R}^{3}$. The application domains considered in the manuscript makes the problem mathematically challenging. We show the uniqueness of the representation in the reduction problem. Then, under some regularity assumptions, we provide an inversion formula to recover the bijection $\Psi$, when it exists, for a given $C$. We also give sufficient conditions for reducibility.

Computationally efficient nonstationary nearest‐neighbor Gaussian process models using data‐driven techniques

AbstractDue to the increased availability of measurements of various geophysical processes, a need has arisen for statistical methods suitable for the analysis of very large nonstationary spatial data sets. The nearest‐neighbor Gaussian process (NNGP) models are one of the latest and most popular Gaussian process‐based models, which reduce computational complexity and memory storage. The Bayesian inference is based on the assumption of a parametric covariance function that is often assumed stationary or known. Given that NNGP models are sensitive in the stationary assumption in comparison to other reduction methods, there is a need to build nonstationary covariance functions within the NNGP models. However, the construction of a nonstationary covariance function and/or matrix may be computationally expensive by itself in the presence of big data. In this paper, we develop an efficient two‐stage approach that deals with nonstationarity and the computational complexity in the presence of a big spatial data set. We propose a new low‐cost data‐driven tree‐structured partitioning technique to divide the spatial region into distinct subregions. Given the partitions, we construct computationally efficient nonstationary covariance functions for NNGP models. We demonstrate the performance of our approach through simulation experiments and an application to the global Total Ozone Matrix Spectrometer (TOMS) data set, in which the proposed approach performs well in terms of both prediction accuracy and computational complexity.

Spatial Statistical Downscaling for Constructing High-Resolution Nature Runs in Global Observing System Simulation Experiments

ABSTRACTObserving system simulation experiments (OSSEs) have been widely used as a rigorous and cost-effective way to guide development of new observing systems, and to evaluate the performance of new data assimilation algorithms. Nature runs (NRs), which are output from deterministic models, play an essential role in building OSSE systems for global atmospheric processes because they are used both to create synthetic observations at high spatial resolution, and to represent the “true” atmosphere against which the forecasts are verified. However, most NRs are generated at resolutions coarser than actual observations from satellite instruments or predictions from data assimilation algorithms. Our goal is to develop a principled statistical downscaling framework to construct high-resolution NRs via conditional simulation from coarse-resolution numerical model output. We use nonstationary spatial covariance function models that have basis function representations to capture spatial variability. This approach not only explicitly addresses the change-of-support problem, but also allows fast computation with large volumes of numerical model output. We also propose a data-driven algorithm to select the required basis functions adaptively, in order to increase the flexibility of our nonstationary covariance function models. In this article we demonstrate these techniques by downscaling a coarse-resolution physical numerical model output at a native resolution of of global surface concentrations to 655,362 equal-area hexagons.

BFDA: A MATLAB Toolbox for Bayesian Functional Data Analysis

We provide a MATLAB toolbox, BFDA, that implements a Bayesian hierarchical model to smooth multiple functional data with the assumptions of the same underlying Gaussian process distribution, a Gaussian process prior for the mean function, and an Inverse-Wishart process prior for the covariance function. This model-based approach can borrow strength from all functional data to increase the smoothing accuracy, as well as estimate the mean-covariance functions simultaneously. An option of approximating the Bayesian inference process using cubic B-spline basis functions is integrated in BFDA, which allows for efficiently dealing with high-dimensional functional data. Examples of using BFDA in various scenarios and conducting follow-up functional regression are provided. The advantages of BFDA include: (1) Simultaneously smooths multiple functional data and estimates the mean-covariance functions in a nonparametric way; (2) flexibly deals with sparse and high-dimensional functional data with stationary and nonstationary covariance functions, and without the requirement of common observation grids; (3) provides accurately smoothed functional data for follow-up analysis.

Efficient Estimation of Non-stationary Spatial Covariance Functions with Application to High-resolution Climate Model Emulation

This research was supported by funding from King Abdullah University of Science and Technology (KAUST). We would like to thank the editor, associate editor, and two anonymous reviewers for their valuable comments.

Semiparametric Modeling with Nonseparable and Nonstationary Spatio-Temporal Covariance Functions and Its Inference

Semiparametric Modeling with Nonseparable and Nonstationary Spatio-Temporal Covariance Functions and Its Inference

Gaussian processes for unconstraining demand

One of the key challenges in revenue management is unconstraining demand data. Existing state of the art single-class unconstraining methods make restrictive assumptions about the form of the underlying demand and can perform poorly when applied to data which breaks these assumptions. In this paper, we propose an unconstraining method that uses Gaussian process (GP) regression. We develop a novel GP model by constructing and implementing a new non-stationary covariance function for the GP which enables it to learn and extrapolate the underlying demand trend. We show that this method can cope with important features of realistic demand data, including nonlinear demand trends, variations in total demand, lengthy periods of constraining, non-exponential inter-arrival times, and discontinuities/changepoints in demand data. In all such circumstances, our results indicate that GPs outperform existing single-class unconstraining methods.

Modeling and emulation of nonstationary Gaussian fields

Geophysical and other natural processes often exhibit nonstationary covariances and this feature is important for statistical models that attempt to emulate the physical process. A convolution-based model is used to represent nonstationary Gaussian processes that allows for variation in the correlation range and the variance of the process across space. We apply this model in two steps: windowed estimates of the covariance function under the assumption of local stationarity and encoded local estimates into a single spatial process model that allows for efficient simulation. We show that nonstationary covariance functions based on the Matérn family can be reproduced by the LatticeKrig (LK) model, a flexible, multi-resolution representation of Gaussian processes. Stationary models based on the Matérn covariance are fit in local windows and these estimates are assembled into a single, global LK model. The LK model is efficient for simulating nonstationary fields even at 105 locations. This work is motivated by the interest in emulating spatial fields derived from numerical model simulations such as Earth system models. We successfully apply these ideas to emulate fields that describe the uncertainty in the pattern scaling of mean summer (JJA) surface temperature from a series of climate model experiments. The spatial covariance structure developed in this paper is not limited to emulation, and could also be used for spatial prediction and conditional simulation for observational data and leverages embarrassingly parallel strategies for computational efficiency.

On a continuous spectral algorithm for simulating non-stationary Gaussian random fields

This paper presents an algorithm for simulating Gaussian random fields with zero mean and non-stationary covariance functions. The simulated field is obtained as a weighted sum of cosine waves with random frequencies and random phases, with weights that depend on the location-specific spectral density associated with the target non-stationary covariance. The applicability and accuracy of the algorithm are illustrated through synthetic examples, in which scalar and vector random fields with non-stationary Gaussian, exponential, Matern or compactly-supported covariance models are simulated.

Local Likelihood Estimation for Covariance Functions with Spatially-Varying Parameters: The convoSPAT Package for R

In spite of the interest in and appeal of convolution-based approaches for nonstationary spatial modeling, off-the-shelf software for model fitting does not as of yet exist. Convolution-based models are highly flexible yet notoriously difficult to fit, even with relatively small data sets. The general lack of pre-packaged options for model fitting makes it difficult to compare new methodology in nonstationary modeling with other existing methods, and as a result most new models are simply compared to stationary models. Using a convolution-based approach, we present a new nonstationary covariance function for spatial Gaussian process models that allows for efficient computing in two ways: first, by representing the spatially-varying parameters via a discrete or mixture model, and second, by estimating the component parameters through a local likelihood approach. In order to make computations for a convolutionbased nonstationary spatial model readily available, this paper also presents and describes the convoSPAT package for R. The nonstationary model is fit to both a synthetic data set and a real data application involving annual precipitation to demonstrate the capabilities of the package.

Modeling nonstationary covariance function with convolution on sphere

The wide use of satellite-based instruments provides measurements in climatology on a global scale, which often have nonstationary covariance structure. The issue of modeling a spatial random fields on sphere which is stationary across longitudes is addressed with a kernel convolution approach. The observed random field is generated by convolving a latent uncorrelated random field with a class of Matérn type kernel functions. By allowing the parameters in the kernel functions to vary with locations, it is possible to generate a flexible class of covariance functions and capture the nonstationary properties. Since the corresponding covariance functions generally do not have a closed form, numerical evaluations are necessary and a pre-computation table is used to speed up the computation. For regular grid data on sphere, the circulant block property of the covariance matrix enables us to use Fast Fourier Transform (FFT) to get its determinant and inverse matrix efficiently. The proposed approach is applied to the Total Ozone Mapping Spectrometer (TOMS) data for illustration.

Regression on the basis of nonstationary Gaussian processes with Bayesian regularization

We consider the regression problem, i.e. prediction of a real valued function. A Gaussian process prior is imposed on the function, and is combined with the training data to obtain predictions for new points. We introduce a Bayesian regularization on parameters of a covariance function of the process, which increases quality of approximation and robustness of the estimation. Also an approach to modeling nonstationary covariance function of a Gaussian process on basis of linear expansion in parametric functional dictionary is proposed. Introducing such a covariance function allows to model functions, which have non-homogeneous behaviour. Combining above features with careful optimization of covariance function parameters results in unified approach, which can be easily implemented and applied. The resulting algorithm is an out of the box solution to regression problems, with no need to tune parameters manually. The effectiveness of the method is demonstrated on various datasets.

A Fused Lasso Approach to Nonstationary Spatial Covariance Estimation

Spatial data are increasing in size and complexity due to technological advances. For an analysis of a large and diverse spatial domain, simplifying assumptions such as stationarity are questionable and standard computational algorithms are inadequate. In this paper, we propose a computationally efficient method to estimate a nonstationary covariance function. We partition the spatial domain into a fine grid of subregions and assign each subregion its own set of spatial covariance parameters. This introduces a large number of parameters and to stabilize the procedure we impose a penalty to spatially smooth the estimates. By penalizing the absolute difference between parameters for adjacent subregions, the solution can be identical for adjacent subregions and thus the method identifies stationary subdomains. To apply the method to large datasets, we use a block composite likelihood which is natural in this setting because it also operates on a partition of the spatial domain. The method is applied to tropospheric ozone in the US, and we find that the spatial covariance on the west coast differs from the rest of the country.

Sampling image segmentations for uncertainty quantification.

In this paper, we introduce a method to automatically produce plausible image segmentation samples from a single expert segmentation. A probability distribution of image segmentation boundaries is defined as a Gaussian process, which leads to segmentations which are spatially coherent and consistent with the presence of salient borders in the image. The proposed approach is computationally efficient, and generates visually plausible samples. The variability between the samples is mainly governed by a parameter which may be correlated with a simple Dice's coefficient, or easily set by the user from the definition of probable regions of interest. The method is extended to the case of several neighboring structures, but also to account for under or over segmentation, and the presence of excluded regions. We also detail a method to sample segmentations with more general non-stationary covariance functions which relies on supervoxels. Furthermore, we compare the generated segmentation samples with several manual clinical segmentations of a brain tumor. Finally, we show how this approach can have useful applications in the field of uncertainty quantification, and an illustration is provided in radiotherapy planning, where segmentation sampling is applied to both the clinical target volume and the organs at risk.

Non-stationary spatio-temporal modeling of traffic-related pollutants in near-road environments

A problem often encountered in environmental epidemiological studies assessing the health effects associated with ambient exposure to air pollution is the spatial misalignment between monitors’ locations and subjects’ actual residential locations. Several strategies have been adopted to circumvent this problem and estimate pollutants concentrations at unsampled sites, including spatial statistical or geostatistical models that rely on the assumption of stationarity to model the spatial dependence in pollution levels. Although computationally convenient, the assumption of stationarity is often untenable for pollutants concentration, particularly in the near-road environment. Building upon the work of Fuentes (2001) and Schmidt et al. (2011), in this paper we present a non-stationary spatio-temporal model for three traffic-related pollutants in a localized near-road environment. Modeling each pollutant separately and independently, we express each pollutant’s concentration as a mixture of two independent spatial processes, each equipped with a non-stationary covariance function with covariates driving the non-stationarity and the mixture weights.