Impact of Ultra-short-term HRV Features in Software Code Sections Complexity Classification

Abstract

Ultra-short term HRV features are becoming increasingly popular due to the fact that they do not need long time periods for their assessment and, therefore, can be used in nearly real-time cognitive load assessment scenarios where the standard 1-min to 5-min time frames are not applicable. Several authors focused on the assessment of the validity of these features by comparing them to the accepted 5-min short term features, showing that the accuracy of these features decreases with the decrease of the analysis window length. However, there is one question that, to the best of our knowledge, has not been fully addressed yet. How does the reduction of the analysis window affect the classification process during cognitive demanding tasks? In this paper we propose the use of 18 different time frames, ranging from 3 minutes to 10 seconds, to extract HRV features from data collected out of 21 subjects during code comprehension tasks. The HRV features are then associated with a code section, gazed during an experiment run, and statistical transformations are computed to built the several datasets, where each section gazed is a sample. A Support Vector Machine (SVM) classifier was trained for each different dataset using a Leave-One-Subject-Out cross-validation procedure, following 3 distinct approaches. The classifier’s goal is to discriminate between low and high complexity code sections analysed by the subjects during the experiment. The F1-Scores ranged from 0.79 to 0.64, indicating that it’s possible to achieve similar, but lower classification results using smaller time frames, with a consistent increase of the variability in the performance evidenced by a higher standard deviation of F1-Scores in the smaller time frames.