Review Rating Prediction with Gaussian Process Classification

Abstract

Many researchers pay attention to training a nonlinear function according to training data because machine learning can represent a more flexible function. One of the approaches is extending a linear function to a nonlinear function with a kernel method. A typical method is Support Vector Machine (SVM), which is a discriminative model. In SVM, a kernel function is defined previously and you need much knowledge on tasks to choose a good kernel function. Hence, the kernel function selection affects the final performance of SVM are many kernel functions are developed according to each task. Another approach is Gaussian process, which is a generative model. Gaussian process estimate describes a distribution over functions and directly search the optimal function in function space. In SVM, a cost function is defined previously and the optimal parameters are searched with respect to training data. Gaussian process and SVM can construct a nonlinear function but their approaches are different. How are Gaussian process and SVM different in practice? In this paper, we evaluate Gaussian process and SVM with respect to review rating prediction, which is one of natural language tasks. Sentiment analysis research denotes some words contribute to sentiment polarity of a sentence strongly. However, deep learning improves the sentiment analysis adding nonlinear feature construction and it is clear that we have to deal with nonlinearity in sentiment analysis. Moreover, review rating prediction is one of multi-class classifications and the prediction needs more flexibility. Finally, when a review is transformed into a numerical vector with a Bag-of-Words model, the review vector has a very high dimension. Hence, the task has to avoid the curse of dimensionality. Kernel trick is the main solution in SVM and a flexible regression function is an additional solution in Gaussian process regression.