Comparison of Selection Criteria for Model Selection of Support Vector Machine on Physiological Data with Inter-Subject Variance

Minho Choi,Jae Jin Jeong

doi:10.3390/app12031749

Abstract

Support vector machines (SVMs) utilize hyper-parameters for classification. Model selection (MS) is an essential step in the construction of the SVM classifier as it involves the identification of the appropriate parameters. Several selection criteria have been proposed for MS, but their usefulness is limited for physiological data exhibiting inter-subject variance (ISV) that makes different characteristics between training and test data. To identify an effective solution for the constraint, this study considered a leave-one-subject-out cross validation-based selection criterion (LSSC) with six well-known selection criteria and compared their effectiveness. Nine classification problems were examined for the comparison, and the MS results of each selection criterion were obtained and analyzed. The results showed that the SVM model selected by the LSSC yielded the highest average classification accuracy among all selection criteria in the nine problems. The average accuracy was 2.96% higher than that obtained with the conventional K-fold cross validation-based selection criterion. In addition, the advantage of the LSSC was more evident for data with larger ISV. Thus, the results of this study can help optimize SVM classifiers for physiological data and are expected to be useful for the analysis of physiological data to develop various medical decision systems.

Highlights

Deep-learning techniques, including convolutional and recurrent neural networks, are being actively researched
Simulations were conducted to analyze the effectiveness of leave-one-subject-out cross validation-based selection criterion (LSSC) and compare it with other well-known selection criteria, namely KCV-based selection criterion (KSC), KSC2, distance between two classes (DBTC), expected square distance ratio (ESDR), xi-alpha bound (XAB), and generalized approximate cross validation (GACV)
These selection criteria were selected in this study because they have been extensively used for Model selection (MS), and their effectiveness has been validated in several studies [25,26,27]

Summary

Introduction

Deep-learning techniques, including convolutional and recurrent neural networks, are being actively researched. Several related studies have reported successful results in many fields [1,2,3,4,5]. The support vector machine (SVM)—one of the well-known machine learning techniques—still constitutes an option for the construction of classification systems [6,7,8]. The SVM has superior generalization ability compared with other classifiers and can be used in instances wherein the training data are not extensive [9,10]. The complexity of the SVM is relatively lower than those of deep-learning techniques [11]. SVM is an effective solution given the limited computational resources

Methods

Results

Discussion

Conclusion