Latent Gaussian Process Research Articles

Complex-valued gaussian process regression for time series analysis

Abstract The construction of synthetic complex-valued signals from real-valued observations is an important part of many time series analysis techniques. The most widely used approach is based on the Hilbert transform, which maps the real-valued signal into its quadrature component. In this paper, we define a probabilistic generalization of this approach. We model the observable real-valued signal as the real part of a latent complex-valued Gaussian process. In order to obtain the appropriate statistical relationship between its real and imaginary parts, we define two new classes of complex-valued covariance functions. Through an analysis of stochastic oscillations, we show that the resulting Gaussian process complex-valued signal provides a better estimate of the instantaneous amplitude and frequency than the established approaches. Furthermore, the complex-valued Gaussian process regression allows to incorporate prior information about the structure in signal and noise and thereby to tailor the analysis to the features of the signal. As a example, we analyze the non-stationary dynamics of brain oscillations in the alpha band, as measured using magneto-encephalography.

A note on using the empirical moment generating function to estimate the variance of nonparametric trend estimates from independent time series replicates

Time series with long-range dependence is often assumed to have been generated from a transformation of a latent stationary Gaussian process with long-memory. Suppose that we have k independent replicates of such time series, which have a common trend Moreover, each series is long range dependent with long-memory parameters which are independently and identically distributed random variables on having an unknown probability distribution. For nonparametric estimation of an important problem is estimation of the variance of the trend estimator. In this note, we propose an algorithm that partially relies on the moment generating function of

Variational Random Function Model for Network Modeling.

Link prediction is a fundamental problem in network modeling. A family of link prediction approaches is to treat network data as an exchangeable array whose entries can be explained by random functions (e.g., block models and Gaussian processes) over latent node factors. Despite their powerful ability in modeling missing links, these models tend to have a large computational complexity and thus are hard to deal with large networks. To address this problem, we develop a novel variational random function model by defining latent Gaussian processes on exchangeable arrays. This model not only inherits the ability of Gaussian process to describe the nonlinear interactions between nodes, but also enjoys significant reduction on computational complexity. To further make the model scalable to large network data, we develop an efficient key-value-free strategy under the map-reduce framework to tremendously reduce the inference time. Experimental results on large network data have demonstrated both the efficacy and efficiency of the proposed method over state-of-the-arts methods in network modeling.

JoineRML: a joint model and software package for time-to-event and multivariate longitudinal outcomes

BackgroundJoint modelling of longitudinal and time-to-event outcomes has received considerable attention over recent years. Commensurate with this has been a rise in statistical software options for fitting these models. However, these tools have generally been limited to a single longitudinal outcome. Here, we describe the classical joint model to the case of multiple longitudinal outcomes, propose a practical algorithm for fitting the models, and demonstrate how to fit the models using a new package for the statistical software platform R, joineRML.ResultsA multivariate linear mixed sub-model is specified for the longitudinal outcomes, and a Cox proportional hazards regression model with time-varying covariates is specified for the event time sub-model. The association between models is captured through a zero-mean multivariate latent Gaussian process. The models are fitted using a Monte Carlo Expectation-Maximisation algorithm, and inferences are based on approximate standard errors from the empirical profile information matrix, which are contrasted to an alternative bootstrap estimation approach. We illustrate the model and software on a real data example for patients with primary biliary cirrhosis with three repeatedly measured biomarkers.ConclusionsAn open-source software package capable of fitting multivariate joint models is available. The underlying algorithm and source code makes use of several methods to increase computational speed.

Gaussian Process Regression and Classification for Probabilistic Damage Assessment of Spatially Distributed Systems

We illustrate how methods for non-parametric regression and classification based on Gaussian Processes can be adapted for inferring the condition state of infrastructure components under spatially distributed stressors. When the stressor is modeled by a random field, observations collected in one location can reduce the uncertainty about the stressor intensity also in other locations. Exact inference is possible when the field is Gaussian and observations are perfect or affected by Gaussian noise. However, often the available observations are binary, as those related to the failure or survival of components, and indicate whether the local field is above or below a threshold whose value may also be uncertain. While no efficient scheme for exact inference is available in that setting, we can perform efficient approximate inference when the field is Gaussian and so is the uncertainty on the threshold value. The mathematical formulation for this problem is analogous to that of classification in machine learning, that can be based on latent Gaussian processes. We show how to formulate the problem and how to adapt deterministic methods, as Laplace’s method and Expectation Propagation, and methods based on random number generation, as Monte Carlo uniform sampling and importance sampling, to perform approximate inference. Our illustrative application is the condition assessment of assets exposed to a seismic event. Under specific assumptions, the seismic demand can be modeled as a Gaussian random field, and measures about the demand and about the survival and failure of assets can be processed globally, to update the risk assessment. Specifically, we evaluate methods for approximate inference and discuss their merits.

Forecasting realized variance measures using time-varying coefficient models

This paper considers the problem of forecasting realized variance measures. These measures are highly persistent estimates of the underlying integrated variance, but are also noisy. Bollerslev, Patton and Quaedvlieg (2016), Journal of Econometrics 192(1), 1–18 exploited this so as to extend the commonly used heterogeneous autoregressive (HAR) by letting the model parameters vary over time depending on the estimated measurement error variances. We propose an alternative specification that allows the autoregressive parameters of HAR models to be driven by a latent Gaussian autoregressive process that may also depend on the estimated measurement error variance. The model parameters are estimated by maximum likelihood using the Kalman filter. Our empirical analysis considers the realized variances of 40 stocks from the S&P 500. Our model based on log variances shows the best overall performance and generates superior forecasts both in terms of a range of different loss functions and for various subsamples of the forecasting period.

Heteroscedastic asymmetric spatial processes

The Gaussian stochastic process is commonly used for modelling time series and geostatistical data. However, the Gaussianity assumption is known to be insufficient or inappropriate in many spatial risk assessment settings. We develop specific non‐Gaussian models to capture the asymmetry and heavy tails of many real‐world spatial datasets. Introducing a general framework for constructing non‐Gaussian spatial processes using transformations of a latent multivariate Gaussian process, we develop a heteroscedastic asymmetric spatial process (HASP) for capturing the non‐Gaussian features of environmental or climatic data, such as heavy tails and skewness. Conditions for this process to be well defined are discussed. The properties of HASP, including marginal moments and the covariance structure, are derived. A Markov chain Monte Carlo procedure is developed for sampling from the posterior distribution. The HASP model is applied to the study of a U.S. nitrogen dioxide concentration dataset.

Calculating Probabilistic Excursion Sets and Related Quantities Using excursions

© 2018, American Statistical Association. All rights reserved. The R software package excursions contains methods for calculating probabilistic excursion sets, contour credible regions, and simultaneous confidence bands for latent Gaussian stochastic processes and fields. It also contains methods for uncertainty quantification of contour maps and computation of Gaussian integrals. This article describes the theoretical and computational methods used in the package. The main functions of the package are introduced and two examples illustrate how the package can be used.

A spatial model for rare binary events

Many predominant spatial methods for binary data use a latent Gaussian process to capture spatial dependence. However, this may not be appropriate for rare data because these methods based on Gaussian processes are asymptotically independent as the event probability goes to zero. In this paper, we propose a method for rare binary data that builds on spatial extreme value theory. We model binary events as exceedances of a max-stable process and show that this construction maintains spatial dependence even as the event probability goes to zero. We compare our model to spatial probit and logistic methods through a simulation study and analysis of a survey of Tamarix ramosissima and Hedysarum scoparium. We find some evidence that for very rare data the max-stable extension provides an improvement in spatial prediction compared to Gaussian models.

Supervised Gaussian Process Latent Variable Model for Hyperspectral Image Classification

Discriminative features are significant for hyper-spectral image (HSI) classification. In this letter, we apply the supervised dimensionality reduction (DR) model termed supervised latent linear Gaussian process latent variable model (SLLGPLVM) for feature extraction. As a semiparametric classification model, the new model has ability in simultaneous feature extraction and classification and demonstrates high classification accuracy with only a small training set. This is therefore suitable for HSI classification. Experimental results on six real HSI data sets show that the proposed SLLGPLVM outperforms several conventional supervised DR models and the support vector machine implemented in the original spectral space.

Generative classification model for categorical data based on latent Gaussian process

In many machine learning applications such as computer-aided diagnosis, gene sequence analysis or natural language processing, categorical data appears. For small-scale data set with high dimensions, since relatively small proportion of possible categorical configurations are covered by training samples, conventional methods based on frequency information such as Dirichlet Compound Multinomial distribution usually runs into problems of over-fitting. Latent gaussian process is an effective bayesian non-parametric technique for categorical data modeling, which was proposed as an unsupervised method to embed unlabelled categorical data into a continuous and low-dimensional space through gaussian process. As a probabilistic generative model, latent gaussian process owns the ability of density estimation. In this paper, we propose a generative classification model as a supervised method for labelled categorical data, in which we use latent gaussian process to estimate the class-conditional densities. Since the complexity of gaussian process model can adapt to the size of training data, our method is able to effectively model small-sale categorical data. Experimental results show that our proposal can achieve better classification performance compared with other classification models for categorical data.

Variational Latent Gaussian Process for Recovering Single-Trial Dynamics from Population Spike Trains

When governed by underlying low-dimensional dynamics, the interdependence of simultaneously recorded populations of neurons can be explained by a small number of shared factors, or a low-dimensional trajectory. Recovering these latent trajectories, particularly from single-trial population recordings, may help us understand the dynamics that drive neural computation. However, due to the biophysical constraints and noise in the spike trains, inferring trajectories from data is a challenging statistical problem in general. Here, we propose a practical and efficient inference method, the variational latent gaussian process (vLGP). The vLGP combines a generative model with a history-dependent point process observation, together with a smoothness prior on the latent trajectories. The vLGP improves on earlier methods for recovering latent trajectories, which assume either observation models inappropriate for point processes or linear dynamics. We compare and validate vLGP on both simulated data sets and population recordings from the primary visual cortex. In the V1 data set, we find that vLGP achieves substantially higher performance than previous methods for predicting omitted spike trains, as well as capturing both the toroidal topology of visual stimuli space and the noise correlation. These results show that vLGP is a robust method with the potential to reveal hidden neural dynamics from large-scale neural recordings.

A max-stable process model for rainfall extremes at different accumulation durations

Abstract A common existing approach to modeling rainfall extremes employs a spatial Bayesian hierarchical model, where latent Gaussian processes are specified on distributional parameters in order to pool spatial information. The data are taken to be conditionally independent given the latent processes, and so spatial dependence arises only through this conditional structure. This methodology can be extended by incorporating an explicit max-stable dependence structure, which therefore produces more realistic spatial surfaces, and removes the assumption of conditional independence. We further extend the max-stable methodology to incorporate the joint modeling of rainfall data at different accumulation durations. We therefore pool information across both space and accumulation duration within a broad framework which includes an explicit specification of spatial dependence. Our model can be used to derive inferences at ungauged sites, and easily incorporates missing values. Our methodology is applied to a dataset of pluviometer records recorded at 182 meteorological stations located on the Central Coast of New South Wales, Australia. For each station, rainfall data are accumulated over 16 different durations ranging from 5 min to 7 days. The model is fitted using Markov chain Monte Carlo simulation, employing auxiliary variables so that exact Bayesian inference can be performed. We present estimated parameters and posterior inferences for isohyetal maps and intensity-duration-frequency curves at selected sites of interest, and compare our inferences with those derived from the standard latent variable model.

Improved Forecasting of Realized Variance Measures

We consider the problem of forecasting realized variance measures. These measures are highly persistent, but also noisy estimates of the underlying integrated variance. Recently, Bollerslev, Patton and Quaedvlieg (2016, Journal of Econometrics, 192, 1-18) exploited this fact to extend the commonly used Heterogeneous Autoregressive (HAR) by letting the model parameters vary over time depending on estimated measurement errors. We propose an alternative specification that allows the autoregressive parameter of the HAR model for volatilities to be driven by a latent Gaussian autoregressive process that may depend on the estimated measurement error. The model is estimated using the Kalman filter. Our analysis considers realized volatilities of 40 stocks from the S&P 500 for three different observation frequencies. Our preferred model provides a better model fit and generates superior forecasts. It consistently outperforms the competing models in terms of different loss functions and for various subsamples of the forecasting period.

Computation of Spatial Gini Coefficients

We address estimation of Gini coefficients for a spatial random field y(s) defined on a lattice, which is assumed to be subordinated to a latent Gaussian process Z(s) via a function G. The marginal distributions of y(s) may have arbitrary shapes and they are location dependent. For we nonparametrically estimate the mean surface and the marginal distribution function which are used to estimate the Gini coefficients. Asymptotic results under short-memory and long-memory auto-correlations in Z are derived. An excerpt from a global total column ozone data set is used for illustration.

Joint modeling of longitudinal drug using pattern and time to first relapse in cocaine dependence treatment data

An important endpoint variable in a cocaine rehabilitation study is the time to first relapse of a patient after the treatment. We propose a joint modeling approach based on functional data analysis to study the relationship between the baseline longitudinal cocaine-use pattern and the interval censored time to first relapse. For the baseline cocaine-use pattern, we consider both self-reported cocaine-use amount trajectories and dichotomized use trajectories. Variations within the generalized longitudinal trajectories are modeled through a latent Gaussian process, which is characterized by a few leading functional principal components. The association between the baseline longitudinal trajectories and the time to first relapse is built upon the latent principal component scores. The mean and the eigenfunctions of the latent Gaussian process as well as the hazard function of time to first relapse are modeled nonparametrically using penalized splines, and the parameters in the joint model are estimated by a Monte Carlo EM algorithm based on Metropolis-Hastings steps. An Akaike information criterion (AIC) based on effective degrees of freedom is proposed to choose the tuning parameters, and a modified empirical information is proposed to estimate the variance-covariance matrix of the estimators.

Likelihood Based Inference and Prediction in Spatio-Temporal Panel Count Models for Urban Crimes

We develop a panel data count model combined with a latent Gaussian spatio-temporal heterogenous state process to analyze monthly severe crimes at the census tract level in Pittsburgh, Pennsylvania. Our data set combines Uniform Crime Reporting data with socio-economic data from the 2000 census. The likelihood of the model is accurately estimated by adapting recently developed efficient importance sampling techniques applicable to high-dimensional spatial models with sparse precision matrices. Our estimation results confirm socio-economic explanations for crime and, foremost, the broken-windows hypothesis, whereby less severe crimes in a region is a leading indicator for severe crimes. In addition to ML parameter estimates, we compute several other statistics of interest for law enforcement such as elasticities (idiosyncratic, total, short-term as well as long-term) of severe crimes w.r.t. less severe crimes, one-month-ahead out-of-sample forecasts, predictive cumulative distribution functions and validation test statistics based on these cdf's.

한국지역 집중호우에 대한 반환주기의 베이지안 모형 분석

Abstract Understanding extreme precipitation events is very important for ood planning purposes. Especially, the r -year return level is a common measure of extreme events. In this paper, we present a spatial analysis ofprecipitation return level using hierarchical Bayesian modeling. For intensity, we model annual maximumdaily precipitations and daily precipitation above a high threshold at 62 stations in Korea with generalizedextreme value(GEV) and generalized Pareto distribution(GPD), respectively. The spatial dependence amongreturn levels is incorporated to the model through a latent Gaussian process of the GEV and GPD modelparameters. We apply the proposed model to precipitation data collected at 62 stations in Korea from 1973to 2011.Keywords: Bayesian analysis, daily precipitation, extremes, generalized extreme value distribution, gener-alized Pareto distribution, return level, spatial process. 1. 서론 최근 지구온난화와 같은전 지구적인기상현상으로 인한 강수의패턴 분석과 예측에 관한 연구가 활발하게 이루어지고 있다. 예를 들어, 대기 중 이산화탄소(CO

Joint Modeling and Clustering Paired Generalized Longitudinal Trajectories With Application to Cocaine Abuse Treatment Data

In a cocaine dependence treatment study, we have paired binary longitudinal trajectories that record the cocaine use patterns of each patient before and after a treatment. To better understand the drug-using behaviors among the patients, we propose a general framework based on functional data analysis to jointly model and cluster these paired non-Gaussian longitudinal trajectories. Our approach assumes that the response variables follow distributions from the exponential family, with the canonical parameters determined by some latent Gaussian processes. To reduce the dimensionality of the latent processes, we express them by a truncated Karhunen-Lóeve (KL) expansion allowing the mean and covariance functions to be different across clusters. We further represent the mean and eigenfunctions functions by flexible spline bases, and determine the orders of the truncated KL expansions using data-driven methods. By treating the cluster membership as a missing value, we cluster the cocaine use trajectories by a likelihood-based approach. The cluster membership and parameter estimates are jointly estimated by a Monte Carlo EM algorithm with Gibbs sampling steps. We discover subgroups of patients with distinct behaviors in terms of overall probability to use, binge verses periodic use pattern, etc. The joint modeling approach also sheds new lights on relating relapse behavior to baseline pattern in each subgroup. Supplementary materials for this article are available online.

On local slope estimation in partial linear models under Gaussian subordination

We address estimation of trend functions and time dependent slope in a partial linear model when the errors are unknown time-dependent functionals of latent Gaussian processes. Asymptotic results are derived under short-memory and long-memory correlations in the data.