A Friend in Need is a Friend Indeed: Investigating the Quality of Training Data from Peers for Auto-generating Empathetic Textual Responses to Non-Sensitive Posts in a Cohort of College Students

Abstract

Towards providing personalized care, digital mental-wellness apps today ask questions to learn about subjects. However, not all subjects using these apps will have mood problems, so they do not need follow-up questions. In this study, we investigate an alternate mechanism to handle such non-sensitive posts (i.e., those not indicating mood problems) in college settings. To do so, we generate and use training data provided by a cohort of peer college students so that responses to non-sensitive posts are contextual, emotionally aware, and empathetic while also being terminal (not asking follow-up questions). Using data from a real mental-wellness app used by students, we identify that AI models trained with our peer-provided dataset generate desirable responses to non-sensitive posts, while models trained with state-of-the-art (Facebook’s) Empathetic Dataset yields responses that ask many follow-up questions, hence giving a perception of being intrusive. We believe that mental wellness apps today must not assume that any subject using these apps has mood problems. Perceptions of intrusiveness (i.e., apps asking many questions) must be a factor in design. We also believe that peer students can provide rich and reliable training datasets for college mental wellness apps, a topic that is not yet explored.