ISO 9001:2015 Quality Management System Requirements and Audit Findings Classification Using Support Vector Machine and Long Short-Term Memory Neural Network: An Optimization Method

Abstract

Generating accurate and timely internal and external audit reports may seem difficult for some auditors due to limited time or expertise in matching the correct clauses of the standard with the textual statement of findings. To overcome this gap, this paper presents the design of text classification models using support vector machine (SVM) and long short-term memory (LSTM) neural network in order to automatically classify audit findings and standard requirements according to text patterns. Specifically, the study explored the optimization of datasets, holdout percentage and vocabulary of learned words called NumWords, then analyzed their capability to predict training accuracy and timeliness performance of the proposed text classification models. The study found that SVM (96.74%) and LSTM (97.54%) were at par with each other in terms of the best training accuracy, although SVM (67.96±17.93 seconds [s]) was found to be significantly faster than LSTM (136.67±96.42 s) in any dataset size. The study proposed optimization formulas that highlight dataset and holdout as predictors of accuracy, while dataset and NumWords as predictors of timeliness. In terms of actual implementation, both classification models were able to accurately classify 20 out of 20 sample audit findings at 1 and 3 s, respectively. Hence, the extent of choosing between the two algorithms depend on the datasets size, learned words, holdout percentage, and workstation speed. This paper is part of a series, which explores the use of Artificial Intelligence (AI) techniques in optimizing the performance of QMS in the context of a state university.