On the optimization of the support vector machine regression hyperparameters setting for gas sensors array applications

Abstract

Support Vector Machine Regression (SVR) has been shown to be more accurate compared to other machine learning techniques that are commonly used for chemical sensors arrays applications. However, the performance of SVR depends strongly on the selection of its hyperparameters. Most of time, researchers in this field rely on trivial grid search methods to find suitable values of SVR hyperparameters by minimizing the cross-validation prediction error. This method is not a practical solution because of the large domain of possible parameter values, which is further exacerbated by the lack of prior knowledge on the data. In this article, we investigate the optimization of SVR hyperparameters by combining the SVR algorithm with a simple algorithm for SVR parameters selection. We begin by studying the influence of each hyperparameter on SVR performance. We then propose the Generalized Pattern Search algorithm (GPS) as a faster alternative to determine these hyperparameters. Finally, we demonstrate that the proposed GPS algorithm, with its simplicity and robustness, gives similar results compared to more complicated alternatives, such as Genetic Algorithms, Simulating Annealing, Bayesian Optimization or Particle Swarm Optimization.