Usability and Emotions of Mental Health Assessment Tools: Comparing Mobile App and Paper-and-Pencil Modalities

Abstract

Users’ experiences in mental health assessment are multifaceted, including their emotional experiences. Yet, studies of mobile apps for psychiatric assessment have centered on diagnostic accuracy and perceived usability, with little consideration of the impact of user emotional experiences. In this study, we focused on users’ perceived usability and emotions and compared the user experience of a paper-and-pencil and an app-based collection of mental health screening questionnaires: EarlyDetect. The System Usability Scale (SUS) and modality-directed emotion questionnaires were administered using paper-and-pencil or iPad. Modality was assigned pseudo-randomly on patients’ first visit at a referral-based mental health clinic. We found that patients assigned to the iPad app reported a significantly higher SUS score than patients assigned to paper-and-pencil, qualified by a modality-by-gender interaction where modality effects were significant for men but not for women. Moreover, enjoyment was positively linked to perceived usability, whereas boredom, frustration, and anxiety were negatively linked to usability. Our findings illustrate the added value of studying user experience applied to psychiatric assessments, where both emotions and gender-specific user experience should be taken into consideration. We further discuss the implications for psychiatric assessments via app versus traditional data collection.