Analysis of Perceived Human Factors and Participants’ Demographics during a Cognitive Assessment Study with a Smartwatch

Abstract

Digital tools have been developed to assess human cognitive functioning. It is unknown to what degree users' cognitive test performance is correlated with their perceived usability and cognitive load induced by interaction with a tool. Moreover, the similarity between user groups in terms of their subjective usability and cognitive load has not been explored adequately despite its potential importance in designing digital cognitive assessment tools for people from diverse background. This paper presents a study of two smartwatch-based cognitive tests to assess participants' attention and working memory. NASA Task Load Index (NASA-TLX) and Mobile App Rating Scale (MARS) questionnaires were used for cognitive load and usability evaluations, respectively. Aesthetics, functionality, and information quality and quantity were the metrics we selected for usability evaluations. Pearson's correlation analysis was performed to investigate the associations and Ward's clustering method was applied for data visualization. Our results showed that participants who received higher scores and longer scoring streak rated functionality of the cognitive tests better. Moreover, information quality and quantity of the tests were rated better by the participants who received longer scoring streak indicating the significant role of test instructions in gaining higher scores. In addition, participants with lower temporal demand received higher scores and faster mean response times. The key findings from the clusters visualized in this paper are: (i) Female and male participants rated their perceived usability and cognitive load completely differently; (ii) A discrepancy was found between participants' perceived performance and their actual scores; (iii) Participants from diverse background rated their perceived usability and cognitive load different from each other.