An Evaluation of the Reliability, Validity and Sensitivity of Three Human Mental Workload Measures Under Different Instructional Conditions in Third-Level Education

Abstract

Although Cognitive Load Theory (CLT) has been researched for many years, it has been criticised for its theoretical clarity and its methodological approach. A crucial issue is the measurement of three types of cognitive load conceived in the theory, and the assessment of overall human cognitive load during learning tasks. This research study is motivated by these issues and it aims to investigate the reliability, validity and sensitivity of three existing self-reporting mental workload instruments, mainly used in Ergonomics, when applied to Education and in particular to the field of Teaching and Learning. A primary research study has been designed and performed in a typical third-level classroom in Computer Science, and the self-reporting mental workload instruments employed are the NASA Task Load Index, the Workload Profile and the Rating Scale Mental Effort. Three instructional design conditions have been designed and employed for the above purposes. The first design condition followed the traditional explicit instruction paradigm whereby a lecturer delivers instructional material mainly using a one-way approach with almost no interactions with students. The second design condition was inspired by the Cognitive Theory of Multimedia Learning whereby the same content, delivered under the first condition, was converted in a multimedia video by following a set of its design principles. The third design condition was an extension of the second condition whereby an inquiry activity was executed after the delivery of the second condition. The empirical evidence gathered in this study suggests that the three selected mental workload measures are highly reliable. Their moderate face validity is in line with the results obtained so far within Ergonomics emphasising and confirming the difficulty in creating optimally valid measures of mental workload. However, the sensitivity of these measures, as achieved in this study, is low, indicating how the three instructional design conditions, as conceived and implemented, do not impose significantly different mental workload levels on learners.