Variable selection by permutation applied in support vector regression models

Abstract

AbstractSupport vector regression (SVR) can be considered as a black box machine‐learning method. Thus, identifying the cause/effect relationship and the synergism between the most important variables is a difficult task. This study demonstrates the potential of two variable selection methods by permutation—subwindow permutation analysis (SPA) and noise incorporated subwindow permutation analysis (NISPA)—to overcome this limitation. The application of these two variable selections in SVR is poorly explored in literature, mainly for regression problems. The algorithms were applied in FTIR (Fourier transform mid infrared spectroscopy) data of crude oil samples to estimate API gravity, kinematic viscosity at 50°C, saturates, aromatics, resins, and asphaltene content. The results were compared to other variable selection methods. SPA and NISPA provided the most accurate models for kinematic viscosity, saturates, and aromatic content. The root‐mean‐squared percentage error of prediction (RMSPEP) of the SPA and NISPA were, respectively, 14.26% and 14.62% for kinematic viscosity, 4.7 wt% and 4.4 wt% for saturates content, and 3.4 wt% and 3.1 wt% for aromatic content. Regarding API prediction, despite obtaining similar accuracy to the other selection methods, SPA produced a more simplified model, using only 3.5% of the 3351 total variables, with RMSEP equal to 1.0 and R2p to 0.981. Therefore, SPA and NISPA, besides obtaining, in general, faster and more accurate and parsimonious models, revealed the most important variables for building the SVR models.