Developing an Advanced Software Requirements Classification Model Using BERT: An Empirical Evaluation Study on Newly Generated Turkish Data

Abstract

Requirements Engineering (RE) is an important step in the whole software development lifecycle. The problem in RE is to determine the class of the software requirements as functional (FR) and non-functional (NFR). Proper and early identification of these requirements is vital for the entire development cycle. On the other hand, manual identification of these classes is a timewaster, and it needs to be automated. Methodically, machine learning (ML) approaches are applied to address this problem. In this study, twenty ML algorithms, such as Naïve Bayes, Rotation Forests, Convolutional Neural Networks, and transformers such as BERT, were used to predict FR and NFR. Any ML algorithm requires a dataset for training. For this goal, we generated a unique Turkish dataset having collected the requirements from real-world software projects with 4600 samples. The generated Turkish dataset was used to assess the performance of the three groups of ML algorithms in terms of F-score and related statistical metrics. In particular, out of 20 ML algorithms, BERTurk was found to be the most successful algorithm for discriminating FR and NFR in terms of a 95% F-score metric. From the FR and NFR identification problem point of view, transformer algorithms show significantly better performances.