A Specialized Probability Density Function for the Input of Mixture of Gaussian Processes

Longbo Zhao,Jinwen Ma

doi:10.1007/978-3-030-01313-4_8

Abstract

Mixture of Gaussian Processes (MGP) is a generative model being powerful and widely used in the fields of machine learning and data mining. However, when we learn this generative model on a given dataset, we should set the probability density function (pdf) of the input in advance. In general, it can be set as a Gaussian distribution. But, for some actual data like time series, this setting or assumption is not reasonable and effective. In this paper, we propose a specialized pdf for the input of MGP model which is a piecewise-defined continuous function with three parts such that the middle part takes the form of a uniform distribution, while the two side parts take the form of Gaussian distribution. This specialized pdf is more consistent with the uniform distribution of the input than the Gaussian pdf. The two tails of the pdf with the form of a Gaussian distribution ensure the effectiveness of the iteration of the hard-cut EM algorithm for MGPs. It demonstrated by the experiments on the simulation and stock datasets that the MGP model with these specialized pdfs can lead to a better result on time series prediction in comparison with the general MGP models as well as the other classical regression methods.

Highlights

Gaussian process(GP) is a powerful model and widely used in machine learning and data mining [1–3]
In order to test the accuracy and effectiveness of the specialized pdf for mixture of Gaussian processes (MGP) model, we carry out several experiments on the simulation and stock datasets. we employ the root mean squared error(RMSE) to measure the prediction accuracy, which is defined as follows: RMSE = Nn=1(yn − yn)2 (15) N
We can obtain that the specialized pdf at both ends of the data in the form of a Gaussian distribution of attenuation, the specialized pdf in the middle of the data is a uniform distribution

Summary

Introduction

Gaussian process(GP) is a powerful model and widely used in machine learning and data mining [1–3]. Many kinds of MGP model have been proposed and can be classified into two main forms: the generative model [7–10] and conditional model [4, 6, 11–13]. When we learn the generative model on a given dataset, we should set the probability density function (pdf) of the input in advance. It can be set as a Gaussian distribution [14–20]. We propose a specialized pdf for the input of the MGP model to solve this problem. For the training of the MGP model, we use the hard-cut EM algorithm [17] as the basic learning framework for parameter estimation.

GP Model

MGP Model

Specialized PDF

Learning Algorithm for the Specialized PDF

The MGP Model of the Specialized PDFs and Its Learning Algorithm

Experimental Results

Simulation Experiments

Prediction on Stock Data

Conclusion