Covariance Kernel Research Articles

Inferring spatially varying animal movement characteristics using a hierarchical continuous-time velocity model.

Understanding the spatial dynamics of animal movement is an essential component of maintaining ecological connectivity, conserving key habitats, and mitigating the impacts of anthropogenic disturbance. Altered movement and migratory patterns are often an early warning sign of the effects of environmental disturbance, and a precursor to population declines. Here, we present a hierarchical Bayesian framework based on Gaussian processes for analysing the spatial characteristics of animal movement. At the heart of our approach is a novel covariance kernel that links the spatially varying parameters of a continuous-time velocity model with GPS locations from multiple individuals. We demonstrate the effectiveness of our framework by first applying it to a synthetic data set and then by analysing telemetry data from the Serengeti wildebeest migration. Through application of our approach, we are able to identify the key pathways of the wildebeest migration as well as revealing the impacts of environmental features on movement behaviour.

Efficient reduced-rank methods for Gaussian processes with eigenfunction expansions

In this work, we introduce a reduced-rank algorithm for Gaussian process regression. Our numerical scheme converts a Gaussian process on a user-specified interval to its Karhunen–Loève expansion, the \(L^2\)-optimal reduced-rank representation. Numerical evaluation of the Karhunen–Loève expansion is performed once during precomputation and involves computing a numerical eigendecomposition of an integral operator whose kernel is the covariance function of the Gaussian process. The Karhunen–Loève expansion is independent of observed data and depends only on the covariance kernel and the size of the interval on which the Gaussian process is defined. The scheme of this paper does not require translation invariance of the covariance kernel. We also introduce a class of fast algorithms for Bayesian fitting of hyperparameters and demonstrate the performance of our algorithms with numerical experiments in one and two dimensions. Extensions to higher dimensions are mathematically straightforward but suffer from the standard curses of high dimensions.

Detecting Anomaly in Chemical Sensors via L1-Kernel-Based Principal Component Analysis

We propose a kernel-PCA-based method to detect anomaly in chemical sensors. We use temporal signals produced by chemical sensors to form vectors to perform the principal component analysis (PCA). We estimate the kernel-covariance matrix of the sensor data and compute the eigenvector corresponding to the largest eigenvalue of the covariance matrix. The anomaly can be detected by comparing the difference between the actual sensor data and the reconstructed data from the dominant eigenvector. In this letter, we introduce a new multiplication-free kernel, which is related to the <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\ell _{1}$</tex-math></inline-formula> -norm for the anomaly detection task. The <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\ell _{1}$</tex-math></inline-formula> -kernel PCA is not only computationally efficient but also energy-efficient because it does not require any actual multiplications during the kernel covariance matrix computation. Our experimental results show that our kernel-PCA method achieves a higher area under curvature score (0.7483) than the baseline regular PCA method (0.7366).

Gaussian process model for the local stellar velocity field from Gaia data release 2

ABSTRACT We model the local stellar velocity field using position and velocity measurements for 4M stars from the second data release of Gaia. We determine the components of the mean or bulk velocity in $\sim 27\, 000$ spatially defined bins. Our assumption is that these quantities constitute a Gaussian process where the correlation between the bulk velocity at different locations is described by a simple covariance function or kernel. We use a sparse Gaussian process algorithm based on inducing points to construct a non-parametric, smooth, and differentiable model for the underlying mean stellar velocity field. We estimate the Oort constants A, B, C, and K and find values in excellent agreement with previous results. Maps of the velocity field within $2\, {\rm kpc}$ of the Sun reveal complicated substructures, which provide clear evidence that the local disc is in a state of disequilibrium. We present the first three-dimensional map of the divergence of the stellar velocity field and identify regions of the disc that may be undergoing compression and rarefaction.

Covariance kernel of linear spectral statistics for half-heavy tailed Wigner matrices

In this paper, we analyze the covariance kernel of the Gaussian process that arises as the limit of fluctuations of linear spectral statistics for Wigner matrices with a few moments. More precisely, the process we study here corresponds to Hermitian matrices with independent entries that have [Formula: see text] moments for [Formula: see text]. We obtain a closed form [Formula: see text]-dependent expression for the covariance of the limiting process resulting from fluctuations of the Stieltjes transform by explicitly integrating the known double Laplace transform integral formula obtained in [F. Benaych-Georges and A. Maltsev, Fluctuations of linear statistics of half-heavy-tailed random matrices, Stochastic Process. Appl. 126(11) (2016) 3331–3352]. We then express the covariance as an integral kernel acting on bounded continuous test functions. The resulting formulation allows us to offer a heuristic interpretation of the impact the typical large eigenvalues of this matrix ensemble have on the covariance structure.

On the eigenvalues associated with the limit null distribution of the Epps-Pulley test of normality

The Shapiro–Wilk test (SW) and the Anderson–Darling test (AD) turned out to be strong procedures for testing for normality. They are joined by a class of tests for normality proposed by Epps and Pulley that, in contrast to SW and AD, have been extended by Baringhaus and Henze to yield easy-to-use affine invariant and universally consistent tests for normality in any dimension. The limit null distribution of the Epps–Pulley test involves a sequences of eigenvalues of a certain integral operator induced by the covariance kernel of a Gaussian process. We solve the associated integral equation and present the corresponding eigenvalues.

Fast Approximate Multioutput Gaussian Processes

Gaussian processes regression models are an appealing machine learning method as they learn expressive nonlinear models from exemplar data with minimal parameter tuning and estimate both the mean and covariance of unseen points. However, cubic computational complexity growth with the number of samples has been a long standing challenge. Training requires the inversion of $N \times N$N×N kernel at every iteration, whereas regression needs computation of an $m \times N$m×N kernel, where $N$N and $m$m are the number of training and test points, respectively. This work demonstrates how approximating the covariance kernel using eigenvalues and functions leads to an approximate Gaussian process with significant reduction in training and regression complexity. Training now requires computing only an $N \times n$N×n eigenfunction matrix and an $n \times n$n×n inverse, where $n$n is a selected number of eigenvalues. Furthermore, regression now only requires an $m \times n$m×n matrix. Finally, in a special case, the hyperparameter optimization is completely independent from the number of training samples. The proposed method can regress over multiple outputs, learn the correlations between them, and estimate their derivatives to any order. The computational complexity reduction, regression capabilities, multioutput correlation learning, and comparison to the state of the art are demonstrated in simulation examples. Finally we show how the proposed approach can be utilized to model real human data.

Coupling of stationary fields with application to arithmetic waves

In this paper we obtain a range of quantitative results of the following type: given two centred Gaussian fields with close covariance kernels we construct a coupling such that the fields are uniformly close on some compact with probability very close to one. As an application we show that it is possible to couple arithmetic random waves so that they converge locally uniformly to the random plane wave and estimate the rate of convergence.

0082 External Validation of an Enhanced Machine Learning Algorithm: Polysomnography-Based Narcolepsy-Like Feature Assessment and Clinician Notification in Routine Sleep Medicine Clinics

Abstract Introduction Polysomnography (PSG) contains quantitative information that, using machine learning (ML) algorithms, may aid the identification of type 1 narcolepsy. This study aimed to evaluate the utility of a previously developed quantitative tool (ML evaluation of PSG data) for detecting narcolepsy (types 1 and 2) in a “real-world” sleep clinic population. Methods Nocturnal PSG studies from a random sample of sleep clinic patients (narcolepsy, n=302; controls, n=21,535) were randomly split (1:1) into a training set and a validation set for algorithm testing. A separate, external PSG dataset was used for additional testing of the final model. Sleep stage probability graphs (hypnodensities) were estimated from PSGs on 15-second epochs using a previously developed convolutional neural network. Feature engineering was applied to hypnodensities to create a feature vector that was used to train a Gaussian process (GP) model to identify patients with a high probability of having narcolepsy. Features could be scaled to the 85th percentile, zero-mean and unit-variance, or unscaled. A recursive feature-elimination scheme was compared with training the GP kernel’s length scale for determining the subset of features that best discriminate narcolepsy and controls. A synthetic minority oversampling technique was applied in combination with random undersampling to balance the distribution of cases and controls in the training set. Several kernels and relevant hyperparameters were evaluated. Model performance (specificity and sensitivity) was examined using receiver operating characteristics with the goal of achieving an area under the curve (AUC) ≥0.80. Results The final GP model used a Matérn 5/2 covariance kernel with the length scale hyperparameter trained to determine the feature subset selection. Input features were normalized to zero-mean and unit-variance. The model had AUC=0.9960 and AUC=0.8014 for classifying narcolepsy in the training and validation sets, respectively. Sensitivity ranged from 73% to 65% when specificity was between 75% and 80%. Final performance evaluation of the model for classifying narcolepsy in an external PSG dataset is ongoing; results will be presented at the congress. Conclusion An ML-based algorithm can offer an objective, sensitive, and specific tool for alerting sleep clinicians about patients at risk for narcolepsy, using nocturnal PSG in general sleep medicine clinics. Support (If Any) Jazz Pharmaceuticals.

Zeros of the i.i.d. Gaussian Laurent Series on an Annulus: Weighted Szegő Kernels and Permanental-Determinantal Point Processes

On an annulus {{mathbb {A}}}_q :={z in {{mathbb {C}}}: q< |z| < 1} with a fixed q in (0, 1), we study a Gaussian analytic function (GAF) and its zero set which defines a point process on {{mathbb {A}}}_q called the zero point process of the GAF. The GAF is defined by the i.i.d. Gaussian Laurent series such that the covariance kernel parameterized by r >0 is identified with the weighted Szegő kernel of {{mathbb {A}}}_q with the weight parameter r studied by McCullough and Shen. The GAF and the zero point process are rotationally invariant and have a symmetry associated with the q-inversion of coordinate z leftrightarrow q/z and the parameter change r leftrightarrow q^2/r. When r=q they are invariant under conformal transformations which preserve {{mathbb {A}}}_q. Conditioning the GAF by adding zeros, new GAFs are induced such that the covariance kernels are also given by the weighted Szegő kernel of McCullough and Shen but the weight parameter r is changed depending on the added zeros. We also prove that the zero point process of the GAF provides a permanental-determinantal point process (PDPP) in which each correlation function is expressed by a permanent multiplied by a determinant. Dependence on r of the unfolded 2-correlation function of the PDPP is studied. If we take the limit q rightarrow 0, a simpler but still non-trivial PDPP is obtained on the unit disk {mathbb {D}}. We observe that the limit PDPP indexed by r in (0, infty ) can be regarded as an interpolation between the determinantal point process (DPP) on {{mathbb {D}}} studied by Peres and Virág (r rightarrow 0) and that DPP of Peres and Virág with a deterministic zero added at the origin (r rightarrow infty ).

Covariance kernels investigation from diffusive wave equations for data assimilation in hydrology

In data assimilation (DA), the estimation of the background error covariance operator is a classical and still open topic. However, this operator is often modeled using empirical information. In order to exploit at best the potential of the knowledge of the physics, the present study proposes a method to derive covariance operators from the underlying equations. In addition, Green’s kernels can be used to model covariance operators and are naturally linked to them. Therefore, Green’s kernels of equations representing physics can provide physically-derived estimates of the background error covariance operator, and also physically-consistent parameters. In this context, the present covariance operators are used in a variational DA (VDA) process of altimetric data to infer bathymetry in the Saint-Venant equations. In order to investigate these new physically-derived covariance operators, the associated VDA results are compared to the VDA results using classical operators with physically-consistent and arbitrary parameters. The physically-derived operators and physically-consistent exponential operator provide better accuracy and faster convergence than empirical operators, especially during the first iterations of the VDA optimization process.

Surrogate Modeling with Gaussian Processes for an Inverse Problem in Polymer Dynamics

When rheological models of polymer blends are used for inverse modeling, they can characterize polymer mixtures from rheological observations. This requires repeated evaluation of potentially expensive rheological models. We explored surrogate models based on Gaussian processes (GP-SM) as a cheaper alternative for describing the rheology of polydisperse binary blends. We used the time-dependent diffusion double reptation (TDD-DR) model as the true model; it takes a 5-dimensional input vector specifying the binary blend as input and yields a function called the relaxation spectrum as output. We used the TDD-DR model to generate training data of different sizes [Formula: see text], via Latin hypercube sampling. The optimal values of the GP-SM hyper-parameters, assuming a separable covariance kernel, were obtained by maximum likelihood estimation. The GP-SM interpolates the training data by design and offers reasonable predictions of relaxation spectra with uncertainty estimates. In general, the accuracy of GP-SMs improves as the size of the training data [Formula: see text] increases, as does the cost for training and prediction. The optimal hyper-parameters were found to be relatively insensitive to [Formula: see text]. Finally, we considered the inverse problem of inferring the structure of the polymer blend from a synthetic dataset generated using the true model. Surprisingly, the solution to the inverse problem obtained using GP-SMs and TDD-DR was qualitatively similar. GP-SMs can be several orders of magnitude cheaper than expensive rheological models, which provides a proof-of-concept validation for using GP-SMs for inverse problems in polymer rheology.

The Gauss hypergeometric covariance kernel for modeling second-order stationary random fields in Euclidean spaces: its compact support, properties and spectral representation

This paper presents a parametric family of compactly-supported positive semidefinite kernels aimed to model the covariance structure of second-order stationary isotropic random fields defined in the d-dimensional Euclidean space. Both the covariance and its spectral density have an analytic expression involving the hypergeometric functions $${}_2F_1$$ and $${}_1F_2$$ , respectively, and four real-valued parameters related to the correlation range, smoothness and shape of the covariance. The presented hypergeometric kernel family contains, as special cases, the spherical, cubic, penta, Askey, generalized Wendland and truncated power covariances and, as asymptotic cases, the Matérn, Laguerre, Tricomi, incomplete gamma and Gaussian covariances, among others. The parameter space of the univariate hypergeometric kernel is identified and its functional properties—continuity, smoothness, transitive upscaling (montée) and downscaling (descente)—are examined. Several sets of sufficient conditions are also derived to obtain valid stationary bivariate and multivariate covariance kernels, characterized by four matrix-valued parameters. Such kernels turn out to be versatile, insofar as the direct and cross-covariances do not necessarily have the same shapes, correlation ranges or behaviors at short scale, thus associated with vector random fields whose components are cross-correlated but have different spatial structures.

Learning Elliptic Partial Differential Equations with Randomized Linear Algebra

Given input–output pairs of an elliptic partial differential equation (PDE) in three dimensions, we derive the first theoretically rigorous scheme for learning the associated Green’s function G. By exploiting the hierarchical low-rank structure of G, we show that one can construct an approximant to G that converges almost surely and achieves a relative error of mathcal {O}(varGamma _epsilon ^{-1/2}log ^3(1/epsilon )epsilon ) using at most mathcal {O}(epsilon ^{-6}log ^4(1/epsilon )) input–output training pairs with high probability, for any 0<epsilon <1. The quantity 0<varGamma _epsilon le 1 characterizes the quality of the training dataset. Along the way, we extend the randomized singular value decomposition algorithm for learning matrices to Hilbert–Schmidt operators and characterize the quality of covariance kernels for PDE learning.

Modeling Neonatal EEG Using Multi-Output Gaussian Processes

Neonatal seizures are sudden events in brain activity with detrimental effects in neurological functions usually related to epileptic fits. Though neonatal seizures can be identified from electroencephalography (EEG), this is a challenging endeavour since expert visual inspection of EEG recordings is time consuming and prone to errors due the data’s nonstationarity and low signal-to-noise ratio. Towards the greater aim of automatic clinical decision making and monitoring, we propose a multi-output Gaussian process (MOGP) framework for neonatal EEG modelling. In particular, our work builds on the multi-output spectral mixture (MOSM) covariance kernel and shows that MOSM outperforms other commonly-used covariance functions in the literature when it comes to data imputation and hyperparameter-based seizure detection. To the best of our knowledge, our work is the first attempt at modelling and classifying neonatal EEG using MOGPs. Our main contributions are: i) the development of an MOGP-based framework for neonatal EEG analysis; ii) the experimental validation of the MOSM covariance kernel on real-world neonatal EEG for data imputation; and iii) the design of features for EEG based on MOSM hyperparameters and their validation for seizure detection (classification) in a patient specific approach.

Bayesian assessments of aeroengine performance with transfer learning

Abstract Aeroengine performance is determined by temperature and pressure profiles along various axial stations within an engine. Given limited sensor measurements, we require a statistically principled approach for inferring these profiles. In this paper we detail a Bayesian methodology for interpolating the spatial temperature or pressure profile at axial stations within an aeroengine. The profile at any given axial station is represented as a spatial Gaussian random field on an annulus, with circumferential variations modelled using a Fourier basis and radial variations modelled with a squared exponential kernel. This Gaussian random field is extended to ingest data from multiple axial measurement planes, with the aim of transferring information across the planes. To facilitate this type of transfer learning, a novel planar covariance kernel is proposed. In the scenario where frequencies comprising the temperature field are unknown, we utilise a sparsity-promoting prior on the frequencies to encourage sparse representations. This easily extends to cases with multiple engine planes whilst accommodating frequency variations between the planes. The main quantity of interest, the spatial area average is readily obtained in closed form. We term this the Bayesian area average and demonstrate how this metric offers far more representative averages than a sector area average---a widely used area averaging approach. Furthermore, the Bayesian area average naturally decomposes the posterior uncertainty into terms characterising insufficient sampling and sensor measurement error respectively. This too provides a significant improvement over prior standard deviation based uncertainty breakdowns.

Spatial-Aware Hyperspectral Nonlinear Unmixing Autoencoder With Endmember Number Estimation

In this article, we develop a novel fully unsupervised autoencoder-based scheme for nonlinear hyperspectral pixel unmixing. A unique approach is derived where high noise and unresponsive pixels are accounted for, by a unique averaging approach based on spatially aware filters built using radial basis function (RBF) kernels. A novel technique is implemented via calculating rank-equivalent kernel covariance matrices in order to estimate the unknown number of endmembers contributing to the data. Utilization of spatial information is done via RBF-based weighted averaging, which is then followed by endmember estimation via K-means clustering. The RBF distances from the cluster centers are determined to measure the position of the mixed pixels in relation to the centers, which is utilized as a preliminary estimation of the abundances. The proposed framework is robust in the presence of unresponsive pixels, while highly versatile working with different nonlinear unmixing models. Extensive numerical tests establish the superiority of the novel approach with respect to state-of-the-art methods.

Structure Parameter Optimized Kernel Based Online Prediction With a Generalized Optimization Strategy for Nonstationary Time Series

In this paper, sparsification techniques aided online prediction algorithms in a reproducing kernel Hilbert space are studied for nonstationary time series. The online prediction algorithms as usual consist of the selection of kernel structure parameters and the kernel weight vector updating. For structure parameters, the kernel dictionary is selected by some sparsification techniques with online selective modeling criteria, and moreover the kernel covariance matrix is intermittently optimized in the light of the covariance matrix adaptation evolution strategy (CMA-ES). Optimizing the real symmetric covariance matrix can not only improve the kernel structure's flexibility by the cross relatedness of the input variables, but also partly alleviate the prediction uncertainty caused by the kernel dictionary selection for nonstationary time series. In order to sufficiently capture the underlying dynamic characteristics in prediction-error time series, a generalized optimization strategy is designed to construct the kernel dictionary sequentially in multiple kernel connection modes. The generalized optimization strategy provides a more self-contained way to construct the entire kernel connections, which enhances the ability to adaptively track the changing dynamic characteristics. Numerical simulations have demonstrated that the proposed approach has superior prediction performance for nonstationary time series.

Quadratic-exponential functionals of Gaussian quantum processes

This paper is concerned with exponential moments of integral-of-quadratic functions of quantum processes with canonical commutation relations of position-momentum type. Such quadratic-exponential functionals (QEFs) arise as robust performance criteria in control problems for open quantum harmonic oscillators (OQHOs) driven by bosonic fields. We develop a randomised representation for the QEF using a Karhunen–Loeve expansion of the quantum process on a bounded time interval over the eigenbasis of its two-point commutator kernel, with noncommuting position-momentum pairs as coefficients. This representation holds regardless of a particular quantum state and employs averaging over an auxiliary classical Gaussian random process whose covariance operator is specified by the commutator kernel. This allows the QEF to be related to the moment-generating functional of the quantum process and computed for multipoint Gaussian states. For stationary Gaussian quantum processes, we establish a frequency-domain formula for the QEF rate in terms of the Fourier transform of the quantum covariance kernel in composition with trigonometric functions. A differential equation is obtained for the QEF rate with respect to the risk sensitivity parameter for its approximation and numerical computation. The QEF is also applied to large deviations and worst-case mean square cost bounds for OQHOs in the presence of statistical uncertainty with a quantum relative entropy description.

Identifiability of covariance kernels in the Gaussian process regression model

Gaussian process regression (GPR) model is a popular nonparametric regression model. In GPR, features of the regression function such as varying degrees of smoothness and periodicities are modeled through combining various covarinace kernels, which are supposed to model certain effects (Bousquet et al., 2011; Gelman et al., 2013). The covariance kernels have unknown parameters which are estimated by the EM-algorithm or Markov Chain Monte Carlo. The estimated parameters are keys to the inference of the features of the regression functions, but identifiability of these parameters has not been investigated. In this paper, we prove identifiability of covariance kernel parameters in two radial basis mixed kernel GPR and radial basis and periodic mixed kernel GPR. We also provide some examples about non-identifiable cases in such mixed kernel GPRs.