Gaussian process models in actuarial science

Machine-Learning Assisted X-Ray Spectroscopy for High- Throughput Characterization of Magnetic Materials

Gaussian process (GP) models are nowadays considered among the state-of-the-art tools in modern dynamic system identification. GP models are probabilistic, non-parametric models based on the principles of Bayesian probability. As a kernel methods they do not try to approximate the modelled system by fitting the parameters of the selected basis functions, but rather by searching for relationships among the measured data. While GP models are Bayesian models they are more robust to overfitting. Moreover, their output is normal distribution, expressed in terms of mean and variance. Due to these features they are used in various fields, e.g. model-based control, time-series prediction, modelling and estimation in engineering applications, etc. But, due to the matrix inversion calculation, whose computationally demand increases with the third power of the number of input data, the amount of training data is limited to at most a few thousand cases. Therefore GP models in principle are not applicable for modelling dynamic systems whose states evolve in time, such as chaotic time-series. In this paper we demonstrate an Evolving GP (EGP) models for predicting chaotic time-series. The EGP is iterative method which adapts model with information obtained from streaming data and concurrently optimizes hyperparameter values. To assess the viability of the EGP an empirical tests were carried out together with a comparative study of various evolving fuzzy methods on a benchmark chaotic time-series MacKey-Glass. The results indicate that the EGP can successfully identify MacKey-Glass chaotic time-series and demonstrate superior performance.

We propose a generalized Gaussian process model (GGPM), which is a unifying framework that encompasses many existing Gaussian process (GP) models, such as GP regression, classification, and counting. In the GGPM framework, the observation likelihood of the GP model is itself parameterized using the exponential family distribution. By deriving approximate inference algorithms for the generalized GP model, we are able to easily apply the same algorithm to all other GP models. Novel GP models are created by changing the parameterization of the likelihood function, which greatly simplifies their creation for task-specific output domains. We also derive a closed-form efficient Taylor approximation for inference on the model, and draw interesting connections with other model-specific closed-form approximations. Finally, using the GGPM, we create several new GP models and show their efficacy in building task-specific GP models for computer vision.

This contribution presents a new development in the design of control system based on evolving Gaussian process (GP) models . GP models provide a probabilistic, nonparametric modelling approach for black-box identification of nonlinear dynamic systems. They can highlight areas of the input space where prediction quality is poor, due to the lack of data or its complexity, by indicating the higher variance around the predicted mean. GP models contain noticeably less coefficients to be optimised than commonly used parametric models. While GP models are Bayesian models, their output is normal distribution, expressed in terms of mean and variance. Latter can be interpreted as a confidence in prediction and used in many fields, especially in control system. Evolving GP model is the concept approach within which various ways of model adaptations can be used. Successful control system needs as much as possible data about process to be controlled . If the prior knowledge about the system to be controlled is scarce or the system varies with time or operating region, this control problem can be solved with an iterative method which adapts model with information obtained with streaming data and concurrently optimises hyperparameter values. This contribution provides: a survey of adaptive control algorithms for dynamic systems described in publications where GP models have been used for control design, a novel and improved closed-loop controller with evolving GP models and an example for the illustration of proposed control algorithm.

This chapter introduces a nonparametric approach for longitudinal data modeling and prediction. This approach is based on the Gaussian process (GP) model. GP is a stochastic process and can be viewed as a distribution over functions with a continuous domain, e.g. time or space. The chapter also introduces the structure of the GP model. It discusses the estimation and prediction methods for the GP model, examines the pairwise GP model. The chapter presents the extension of a single output GP model to the general Multiple Output Gaussian Process (MOGP) model, which plays a critical role in longitudinal data prediction. It discusses the time-to-failure distribution based on the MOGP model. The chapter provides the basic structure of MOGP and explains the MOGP based prediction method. The Gaussian process, also known as Kriging method, is a very flexible yet effective nonparametric model to describe smooth functional data.

Control system based on evolving Gaussian process (GP) models is an example of self-learning closed-loop control system. It is meant for closed-loop control of dynamic systems where not much prior knowledge exists or where systems dynamics varies with time or operating region. GP models are non-parametric black-box models which represent a new method for system identification. GP models differ from most other frequently used black-box identification approaches as they do not try to approximate the modeled system by fitting the parameters of the selected basis functions, but rather search for the relationships among measured data. While GP models are Bayesian models, their output is normal distribution, expressed in terms of mean and variance. Latter can be interpreted as a confidence in prediction and used in many fields, especially in control system. Successful control system needs as much as possible data about process to be controlled. If the prior knowledge about the system to be controlled is scarce or the system varies with time or operating region, this control problem can be solved with an iterative method which adapts model with information obtained with streaming data and concurrently optimizes hyperparameter values. While that kind of method for GP models does not yet exist, concepts for evolving GP models and control system based on evolving GP models are proposed in this paper. It is flexible approach within which various ways of model adaptations can be used. One of those possibilities is illustrated with a control of a benchmark problem.

Bayesian optimization with Gaussian Process (GP) models has been proposed for analog synthesis since it is efficient for the optimizations of expensive black-box functions. However, the computational cost for training and prediction of Gaussian process models are O(N <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">3</sup> ) and O(N <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> ), respectively, where N is the number of data points. The overhead of the Gaussian process modeling would not be negligible as N is relatively large. Recently, a Bayesian optimization approach using neural network has been proposed to address this problem. It reduces the computational cost of training and prediction of Gaussian process models to O(N) and O(1), respectively. However, reducing the infinite-dimensional kernel to finite-dimensional kernel using neural network mapping would weaken the characterization ability of Gaussian process. In this paper, we propose a novel Bayesian optimization approach using Sparse Pseudo-input Gaussian Process (SPGP). The idea is to use M <; N so-called inducing points to build a sparse Gaussian process model to approximate the conventional exact Gaussian process model. Without the need to sacrifice the modeling ability of the surrogate model, it also reduces the computational cost of both training and prediction to O(N) and O(1), respectively. Several experiments were provided to demonstrate the efficiency of the proposed approach.

In this paper we introduce Gaussian Process (GP) models for music genre classification. Gaussian Processes are widely used for various regression and classification tasks, but there are relatively few studies where GPs are applied in the audio signal processing systems. The GP models are non-parametric discriminative classifiers similar to the well known SVMs in terms of usage. In contrast to SVMs, however, GP models produce truly probabilistic output and allow for kernel function parameters to be learned from the training data. In this work we compare the performance of GP models and SVMs as music genre classifiers using the ISMIR 2004 database. Audio preprocessing is the same for both cases and is based on Constant-Q spectrograms. The experimental results using linear as well as exponential kernel functions and different amounts of training data show that GP models always outperform SVMs with up to 5.6% absolute difference in the classification accuracy.

Elevated temperatures limit the peak performance of systems because of frequent interventions by thermal throttling. Non-uniform thermal states across system nodes also cause performance variation within seemingly equivalent nodes leading to significant degradation of overall performance. In this paper we present a framework for creating a lightweight thermal prediction system suitable for run-time management decisions. We pursue two avenues to explore optimized lightweight thermal predictors. First, we use feature selection algorithms to improve the performance of previously designed machine learning methods. Second, we develop alternative methods using neural network and linear regression-based methods to perform a comprehensive comparative study of prediction methods. We show that our optimized models achieve improved performance with better prediction accuracy and lower overhead as compared with the Gaussian process model proposed previously. Specifically we present a reduced version of the Gaussian process model, a neural network–based model, and a linear regression–based model. Using the optimization methods, we are able to reduce the average prediction errors in the Gaussian process from $4.2^\circ$ C to $2.9^\circ$ C. We also show that the newly developed models using neural network and Lasso linear regression have average prediction errors of $2.9^\circ$ C and $3.8^\circ$ C respectively. The prediction overheads are 0.22, 0.097, and 0.026 ms per prediction for reduced Gaussian process, neural network, and Lasso linear regression models, respectively, compared with 0.57 ms per prediction for the previous Gaussian process model. We have implemented our proposed thermal prediction models on a two-node system configuration to help identify the optimal task placement. The task placement identified by the models reduces the average system temperature by up to $11.9^\circ$ C without any performance degradation. Furthermore, these models respectively achieve 75, 82.5, and 74.17 percent success rates in correctly pointing to those task placements with better thermal response, compared with 72.5 percent success for the original model in achieving the same objective. Finally, we extended our analysis to a 16-node system and we were able to train models and execute them in real time to guide task migration and achieve on average 17 percent reduction in the overall system cooling power.

Gaussian process (GP) models based model predictive control (MPC) structure for cooperative motion planning of Unmanned Aerial and Ground Vehicle System (UAGVS) is proposed in this article. The GP models are firstly trained to describe the dynamics of UAVs and UGVs with their uncertainties. Stochastic optimization problems are designed for motion planning and controlling based on the GP models' probabilistic predictions. And for the necessary interactions among agents, the predicted motion information is also communicated with each other, so that they can obtain others' future planned behaviors. Simulation results show that the proposed method exhibit promising effects of motion planning for UAVGS.

In the 1960s E. Fama developed the efficient market hypothesis (EMH) which asserts that the financial market is efficient if its prices are formed on the basis of all publicly available information. That means technical analysis cannot be used to predict and beat the market. Since then, it was widely examined and was mostly accepted by mathematicians and financial engineers. However, the predictability of financial-market returns remains an open problem and is discussed in many publications. Usually, it is concluded that a model able to predict financial returns should adapt to market changes quickly and catch local dependencies in price movements. The Bayesian vector autoregression (BVAR) models, support vector machines (SVM) and some other were already applied to financial data quite succesfully. Gaussian process (GP) models are emerging non-parametric Bayesian models and in this paper we test their applicability to financial data. GP model is fitted to daily data from U.S. commodity markets. For a comparison BVAR model and benchmark model that is commonly used in todays financial mathematics are chosen. The results indicate that GP models are applicable to financial data as well as BVAR models.

Systems of Gaussian process models for directed chains of solvers

Standard bearing fault detection features are shown to be ineffective for estimating bearings remaining useful life (RUL). Addressing this issue, in this paper we propose an approach for bearing fault prognostics based on features describing the statistical complexity of the envelope of the generated vibrations and a set of Gaussian process (GP) models. The proposed feature set exhibits continuous trend which can be directly related to the deterioration of bearing condition. Gaussian process models are non-parametric black-box models which differ from most other frequently used black-box identification approaches as they search for the relationships among measured data rather than trying to approximate the modeled system by fitting the parameters of the selected basis functions. Their output is normal distribution, expressed in terms of mean and variance, which can be interpreted as a confidence in prediction. In this paper the GP models are used for filtering noisy features and estimating the RUL based on filtered features. The proposed approach was evaluated on the data set provided for the IEEE PHM 2012 Prognostic Challenge.

Quantitative prediction of subjective pain intensity from whole-brain fMRI data using Gaussian processes

The aim was to assess mathematically the nature of a skin permeability dataset and to determine the utility of Gaussian processes in developing a predictive model for skin permeability, comparing it with existing methods for deriving predictive models. Principal component analysis was carried out in order to determine the nature of the dataset. MatLab software was used to assess the performance of Gaussian process, single linear networks (SLN) and quantitative structure-permeability relationships (QSPRs) using a range of statistical measures. Principal component analysis showed that the dataset is inherently non-linear. The Gaussian process model yielded a predictive model that provides a significantly more accurate estimate of skin absorption than previous models, particularly QSPRs (which were consistently worse than Gaussian process or SLN models), and does so across a wider range of molecular properties. Gaussian process models appear particularly capable of providing excellent predictions where previous studies have shown QSPRs to fail, such as where penetrants have high log P and high molecular weight. A non-linear approach was more appropriate than QSPRs or SLNs for the analysis of the dataset employed herein, as the prediction and confidence values in the prediction given by the Gaussian process are better than with other methods examined. Gaussian process provides a novel way of analysing skin absorption data that is substantially more accurate, statistically robust and reflective of our empirical understanding of skin absorption than the QSPR methods so far applied to skin absorption.