Hyperparameter Tuning of Machine Learning Algorithms Using Response Surface Methodology: A Case Study of ANN, SVM, and DBN

Jirachai Buddhakulsomsiri,Kuei-Hu Chang,Kwanluck Thiwa-Anont,Warut Pannakkong,Parthana Parthanadee,Kasidit Singthong

doi:10.1155/2022/8513719

Jirachai Buddhakulsomsiri, Kuei-Hu Chang + Show 4 more

Open Access

https://doi.org/10.1155/2022/8513719

Copy DOI

Abstract

This study applies response surface methodology (RSM) to the hyperparameter fine-tuning of three machine learning (ML) algorithms: artificial neural network (ANN), support vector machine (SVM), and deep belief network (DBN). The purpose is to demonstrate RSM effectiveness in maintaining ML algorithm performance while reducing the number of runs required to reach effective hyperparameter settings in comparison with the commonly used grid search (GS). The ML algorithms are applied to a case study dataset from a food producer in Thailand. The objective is to predict a raw material quality measured on a numerical scale. K-fold cross-validation is performed to ensure that the ML algorithm performance is robust to the data partitioning process in the training, validation, and testing sets. The mean absolute error (MAE) of the validation set is used as the prediction accuracy measurement. The reliability of the hyperparameter values from GS and RSM is evaluated using confirmation runs. Statistical analysis shows that (1) the prediction accuracy of the three ML algorithms tuned by GS and RSM is similar, (2) hyperparameter settings from GS are 80% reliable for ANN and DBN, and settings from RSM are 90% and 100% reliable for ANN and DBN, respectively, and (3) savings in the number of runs required by RSM over GS are 97.79%, 97.81%, and 80.69% for ANN, SVM, and DBN, respectively.

Highlights

Nowadays, machine learning (ML) algorithms have become an important part of various industries
response surface methodology (RSM) is compared with grid search (GS) on three ML algorithms: artificial neural network (ANN), support vector machine (SVM), and deep belief network (DBN)
E dataset used in the computational test is from an industrial user in the food industry. e purpose of the analysis is to predict the quality of the raw material in the production process. e mean absolute error (MAE) is considered a key measure. e MAE from the validation set is used to evaluate the prediction performance between the two hyperparameter tuning processes

Summary

Introduction

Machine learning (ML) algorithms have become an important part of various industries. ML can be used to reduce the labor cost, human error, and number of product defects or to increase the production rate. Another advantage is that ML algorithms can handle a large amount of input data for model training [1]. E main reason for ML algorithms’ growing use is that ML algorithms can improve productivity and efficiency by automating them in the usage environment [1, 2]. ML algorithms can learn from previous experience by discovering patterns in existing data and using those patterns to develop and/or improve their knowledge over time [3]. ML algorithms can learn from previous experience by discovering patterns in existing data and using those patterns to develop and/or improve their knowledge over time [3]. ese benefits of ML algorithms can lead to business revenue and growth

Results

Discussion

Conclusion