Multi-objective adaptive differential evolution for SVM/SVR hyperparameters selection

Abstract

Parameters Selection Problem (PSP) is a relevant and complex optimization issue in Support Vector Machine (SVM) and Support Vector Regression (SVR), looking for obtaining an optimal set of hyperparameters. In our case, the optimization problem is addressed to obtain models that minimize the number of support vectors and maximize generalization capacity. However, to obtain accurate and low complexity solutions, defining an adequate kernel function and the SVM/SVR’s hyperparameters are necessary, which currently represents a relevant research topic. To tackle this problem, this work proposes a multi-objective metaheuristic named Adaptive Parameter control with Mutant Tournament Multi-Objective Differential Evolution (APMT-MODE). Its performance is first tested in a series of benchmarks for classification and regression problems using simple kernels such as Gaussian and polynomial kernels. In both cases, the APMT-MODE is able to yield more precise and more straightforward solutions using simple kernels. Then, the approach is used on a real case study to create a welding bead depth and width SVR models for a Gas Metal Arc Welding (GMAW) process. Additionally, a study on kernel functions was developed in terms of computational effort, aiming to assess its performance for embedded systems applications.