Student-t Process Research Articles

A multivariate student-t process model for dependent tail-weighted degradation data

Traditionally, the Gaussian assumption, implied by the Wiener process, is widely admitted for modeling degradation processes. However, when degradation data exhibit heavy tails, this assumption is not suitable. To overcome this limitation, this article proposes a novel class of tail-weighted multivariate degradation model, which is built upon Student-t process. The model is able to account for both between-unit variability and process dependency, while allowing the adjustment of tail heaviness through tuning the parameter of the degree of freedom. For reliability assessment, we derive the system reliability function and present an efficient Monte Carlo method for its evaluation. Further, we introduce an expectation–maximization algorithm for parameter estimation and design a bootstrap method for interval estimation. Comprehensive simulation studies are conducted to validate the effectiveness of the inference method. Finally, the proposed methodology is applied to analyze two real-world degradation datasets.

Sparse Variational Student-t Processes

The theory of Bayesian learning incorporates the use of Student-t Processes to model heavy-tailed distributions and datasets with outliers. However, despite Student-t Processes having a similar computational complexity as Gaussian Processes, there has been limited emphasis on the sparse representation of this model. This is mainly due to the increased difficulty in modeling and computation compared to previous sparse Gaussian Processes. Our motivation is to address the need for a sparse representation framework that reduces computational complexity, allowing Student-t Processes to be more flexible for real-world datasets. To achieve this, we leverage the conditional distribution of Student-t Processes to introduce sparse inducing points. Bayesian methods and variational inference are then utilized to derive a well-defined lower bound, facilitating more efficient optimization of our model through stochastic gradient descent. We propose two methods for computing the variational lower bound, one utilizing Monte Carlo sampling and the other employing Jensen's inequality to compute the KL regularization term in the loss function. We propose adopting these approaches as viable alternatives to Gaussian processes when the data might contain outliers or exhibit heavy-tailed behavior, and we provide specific recommendations for their applicability. We evaluate the two proposed approaches on various synthetic and real-world datasets from UCI and Kaggle, demonstrating their effectiveness compared to baseline methods in terms of computational complexity and accuracy, as well as their robustness to outliers.

Objective Bayesian analysis for geostatistical Student-t processes

ABSTRACT The choice of the prior distribution is a key aspect of Bayesian analysis. The spatial Student-t regression model poses some challenges when eliciting priors. It is well known that the propriety of the posterior distribution over objective priors is not always guaranteed, whereas the use of proper prior distributions may dominate and bias the posterior analysis. In this paper, we show the conditions under which our proposed reference prior and the two introduced Jeffreys priors yield to a proper posterior distribution. Simulation studies and a real data application are used in order to evaluate the performance of the reference prior.

Calibration of uncertainty in the active learning of machine learning force fields

FFLUX is a machine learning force field that uses the maximum expected prediction error (MEPE) active learning algorithm to improve the efficiency of model training. MEPE uses the predictive uncertainty of a Gaussian process (GP) to balance exploration and exploitation when selecting the next training sample. However, the predictive uncertainty of a GP is unlikely to be accurate or precise immediately after training. We hypothesize that calibrating the uncertainty quantification within MEPE will improve active learning performance. We develop and test two methods to improve uncertainty estimates: post-hoc calibration of predictive uncertainty using the CRUDE algorithm, and replacing the GP with a student-t process. We investigate the impact of these methods on MEPE for single sample and batch sample active learning. Our findings suggest that post-hoc calibration does not improve the performance of active learning using the MEPE method. However, we do find that the student-t process can outperform active learning strategies and random sampling using a GP if the training set is sufficiently large.

Outlier-Robust Surrogate Modeling of Ion-Solid Interaction Simulations.

Data for complex plasma-wall interactions require long-running and expensive computer simulations. Furthermore, the number of input parameters is large, which results in low coverage of the (physical) parameter space. Unpredictable occasions of outliers create a need to conduct the exploration of this multi-dimensional space using robust analysis tools. We restate the Gaussian process (GP) method as a Bayesian adaptive exploration method for establishing surrogate surfaces in the variables of interest. On this basis, we expand the analysis by the Student-t process (TP) method in order to improve the robustness of the result with respect to outliers. The most obvious difference between both methods shows up in the marginal likelihood for the hyperparameters of the covariance function, where the TP method features a broader marginal probability distribution in the presence of outliers. Eventually, we provide first investigations, with a mixture likelihood of two Gaussians within a Gaussian process ansatz for describing either outlier or non-outlier behavior. The parameters of the two Gaussians are set such that the mixture likelihood resembles the shape of a Student-t likelihood.

Prediction of Bus Passenger Traffic using Gaussian Process Regression.

The paper summarizes the design and implementation of a passenger traffic prediction model, based on Gaussian Process Regression (GPR). Passenger traffic analysis is the present day requirement for proper bus scheduling and traffic management to improve the efficiency and passenger comfort. Bayesian analysis uses statistical modelling to recursively estimate new data from existing data. GPR is a fully Bayesian process model, which is developed using PyMC3 with Theano as backend. The passenger data is modelled as a Poisson process so that the prior for designing the GP regression model is a Gamma distributed function. It is observed that the proposed GP based regression method outperforms the existing methods like Student-t process model and Kernel Ridge Regression (KRR) process.

Evaluating Gaussian process metamodels and sequential designs for noisy level set estimation

We consider the problem of learning the level set for which a noisy black-box function exceeds a given threshold. To efficiently reconstruct the level set, we investigate Gaussian process (GP) metamodels. Our focus is on strongly stochastic simulators, in particular with heavy-tailed simulation noise and low signal-to-noise ratio. To guard against noise misspecification, we assess the performance of three variants: (i) GPs with Student-t observations; (ii) Student-t processes (TPs); and (iii) classification GPs modeling the sign of the response. In conjunction with these metamodels, we analyze several acquisition functions for guiding the sequential experimental designs, extending existing stepwise uncertainty reduction criteria to the stochastic contour-finding context. This also motivates our development of (approximate) updating formulas to efficiently compute such acquisition functions. Our schemes are benchmarked by using a variety of synthetic experiments in 1–6 dimensions. We also consider an application of level set estimation for determining the optimal exercise policy of Bermudan options in finance.

Student's T-process Regression on the Space of Probability Density Functions

Student's T-process Regression on the Space of Probability Density Functions

Altering Gaussian process to Student-t process for maximum distribution construction

Gaussian process (GP) regression is widely used to find the extreme of a black-box function by iteratively approximating an objective function when new evaluation obtained. Such evaluation is usually made by optimising a given acquisition function. However, for non-parametric Bayesian optimisation, the extreme of the objective function is not a deterministic value, but a random variant with distribution. We call such distribution the maximum distribution which is generally non-analytical. To construct such maximum distribution, traditional GP regression method by optimising an acquisition function is computational cost as the GP model has a cubic computation of training data. Moreover, the introduction of acquisition function brings extra hyperparameters which made the optimisation more complicated. Recently, inspired by the idea of Sequential Monte Carlo method and its application in Bayesian optimisation, a Monte Carlo alike method is proposed to approximate the maximum distribution with weighted samples. Alternative to the method on GP model, we construct the maximum distribution within the framework of Student-t process (TP) which considers more uncertainties from the training data. Toy examples and real data experiment show the TP-based Monte Carlo maximum distribution has a competitive performance to the GP-based method.

Projecting Flood-Inducing Precipitation with a Bayesian Analogue Model

The hazard of pluvial flooding is largely influenced by the spatial and temporal dependence characteristics of precipitation. When extreme precipitation possesses strong spatial dependence, the risk of flooding is amplified due to catchment factors such as topography that cause runoff accumulation. Temporal dependence can also increase flood risk as storm water drainage systems operating at capacity can be overwhelmed by heavy precipitation occurring over multiple days. While transformed Gaussian processes are common choices for modeling precipitation, their weak tail dependence may lead to underestimation of flood risk. Extreme value models such as the generalized Pareto processes for threshold exceedances and max-stable models are attractive alternatives, but are difficult to fit when the number of observation sites is large, and are of little use for modeling the bulk of the distribution, which may also be of interest to water management planners. While the atmospheric dynamics governing precipitation are complex and difficult to fully incorporate into a parsimonious statistical model, non-mechanistic analogue methods that approximate those dynamics have proven to be promising approaches to capturing the temporal dependence of precipitation. In this paper, we present a Bayesian analogue method that leverages large, synoptic-scale atmospheric patterns to make precipitation forecasts. Changing spatial dependence across varying intensities is modeled as a mixture of spatial Student-t processes that can accommodate both strong and weak tail dependence. The proposed model demonstrates improved performance at capturing the distribution of extreme precipitation over Community Atmosphere Model (CAM) 5.2 forecasts.Supplementary materials accompanying this paper appear online.

Correction to: Multivariate Gaussian and Student-t process regression for multi-output prediction

Correction to: Multivariate Gaussian and Student-t process regression for multi-output prediction

Multivariate Gaussian and Student-t process regression for multi-output prediction

Gaussian process model for vector-valued function has been shown to be useful for multi-output prediction. The existing method for this model is to reformulate the matrix-variate Gaussian distribution as a multivariate normal distribution. Although it is effective in many cases, reformulation is not always workable and is difficult to apply to other distributions because not all matrix-variate distributions can be transformed to respective multivariate distributions, such as the case for matrix-variate Student-t distribution. In this paper, we propose a unified framework which is used not only to introduce a novel multivariate Student-t process regression model (MV-TPR) for multi-output prediction, but also to reformulate the multivariate Gaussian process regression (MV-GPR) that overcomes some limitations of the existing methods. Both MV-GPR and MV-TPR have closed-form expressions for the marginal likelihoods and predictive distributions under this unified framework and thus can adopt the same optimization approaches as used in the conventional GPR. The usefulness of the proposed methods is illustrated through several simulated and real-data examples. In particular, we verify empirically that MV-TPR has superiority for the datasets considered, including air quality prediction and bike rent prediction. At last, the proposed methods are shown to produce profitable investment strategies in the stock markets.

Student-t Processes for Degradation Analysis

Stochastic processes are widely used to analyze degradation data, and the Gaussian process is a particularly common one. In this article, we propose a robust statistical model using a Student-t process to assess the lifetime information of highly reliable products. This model is statistically plausible and demonstrates a substantially improved fit when applied to real data. A computationally accurate approach is proposed to calculate the first-passage-time density function of the Student-t degradation-based process; related properties are investigated as well. In addition, this article provides parameter estimation using the EM-type algorithm and a simple model-checking procedure to evaluate the appropriateness of the model assumptions. Several case studies are performed to demonstrate the flexibility and applicability of the proposed model with random effects and explanatory variables. Technical details, datasets, and R codes are available as supplementary materials.

Robust functional regression model for marginal mean and subject-specific inferences.

We introduce flexible robust functional regression models, using various heavy-tailed processes, including a Student t-process. We propose efficient algorithms in estimating parameters for the marginal mean inferences and in predicting conditional means as well as interpolation and extrapolation for the subject-specific inferences. We develop bootstrap prediction intervals (PIs) for conditional mean curves. Numerical studies show that the proposed model provides a robust approach against data contamination or distribution misspecification, and the proposed PIs maintain the nominal confidence levels. A real data application is presented as an illustrative example.

Robust and Trend-following Kalman Smoothers using Student's t

We propose two nonlinear Kalman smoothers that rely on Student's t distributions. The t-Robust smoother, finds the maximum a posteriori likelihood (MAP) solution for Gaussian process noise and Student's t observation noise, and is extremely robust against outliers, outperforming the recently proposed l1-Laplace smoother in extreme situations (e.g. 50% or more outliers). The second estimator, which we call the t-Trend smoother, is able to follow sudden changes in the process model, and is derived as a MAP solver for a model with Student's t-process noise and Gaussian observation noise. We design specialized methods to solve both problems which exploit the special structure of the Student's t-distribution, and provide a convergence theory. Both smoothers can be implemented with only minor modifications to an existing L2 smoother implementation. Numerical results for linear and nonlinear models illustrating both robust and fast tracking applications are presented.

Robust non‐linear mixed modelling of longitudinal PSA levels after prostate cancer treatment

The objective of this study was to develop a robust non-linear mixed model for prostate-specific antigen (PSA) measurements after a high-intensity focused ultrasound (HIFU) treatment for prostate cancer. The characteristics of these data are the presence of outlying values and non-normal random effects. A numerical study proved that parameter estimates can be biased if these characteristics are not taken into account. The intra-patient variability was described by a Student-t distribution and Dirichlet process priors were assumed for non-normal random effects; a process that limited the bias and provided more efficient parameter estimates than a classical mixed model with normal residuals and random effects. It was applied to the determination of the best dynamic PSA criterion for the diagnosis of prostate cancer recurrence, but could be used in studies that rely on PSA data to improve prognosis or compare treatment efficiencies and also with other longitudinal biomarkers that, such as PSA, present outlying values and non-normal random effects.

On some results of Cufaro Petroni about Student t-processes

This paper deals with Student t-processes as studied in Cufaro Petroni (2007 J. Phys. A. Math. Theor. 40 2227–50). We prove and extend some conjectures expressed by Cufaro Petroni about the asymptotical behavior of a Student t-process and the expansion of its density. First, the explicit asymptotic behavior of any real positive convolution power of a Student t-density with any real positive degrees of freedom is given in the multivariate case; then the integer convolution power of a Student t-distribution with odd degrees of freedom is shown to be a convex combination of Student t-densities with odd degrees of freedom. At last, we show that this result does not extend to the case of non-integer convolution powers.

Of Parameters and Principles: Producing Theory in Twentieth Century Physics and Chemistry

We propose two nonlinear Kalman smoothers that rely on Student's t distributions. The t-Robust smoother, finds the maximum a posteriori likelihood (MAP) solution for Gaussian process noise and Student's t observation noise, and is extremely robust against outliers, outperforming the recently proposed l1-Laplace smoother in extreme situations (e.g. 50% or more outliers). The second estimator, which we call the t-Trend smoother, is able to follow sudden changes in the process model, and is derived as a MAP solver for a model with Student's t-process noise and Gaussian observation noise. We design specialized methods to solve both problems which exploit the special structure of the Student's t-distribution, and provide a convergence theory. Both smoothers can be implemented with only minor modifications to an existing L2 smoother implementation. Numerical results for linear and nonlinear models illustrating both robust and fast tracking applications are presented.