Bayesian optimization of multiclass SVM for efficient diagnosis of erythemato-squamous diseases

Abstract

Recently, Bayesian Optimization (BO) has emerged as an efficient technique for adjusting the hyperparameters of machine learning models. BO approach develops an alternative mathematical function to efficiently optimize the computation-intensive functions. In this paper, we demonstrate the utility of this approach in hyperparameter optimizations and feature selection for the multiclass support vector machine (SVM). The efficiency of the proposed BO-SVM hybrid model was evaluated in the differential diagnosis of the erythemato-squamous diseases (ESDs) dataset from UCI machine learning repository. The dataset contains the results of clinical and histopathological tests for six different skin diseases. The multiclass problem has been manipulated using four different Error-Correcting Output Codes (ECOC) coding schemes: one-versus-all, binary complete, one-versus-one, and ternary complete. BO has been implemented using the Gaussian process (GP) model with Matérn covariance kernel and expected improvement acquisition function. Our experimental results show that the advantage of the multiclass BO-SVM with 100% and 99.07% training and test classification accuracies respectively. Some basic and practical procedures in model development and evaluation such as normalization, cross-validation, decimal to binary mask conversion, feature selection and a comparison between predictive power of the clinical and histopathological subsets are also referred to.