ABSTRACT The current study presents the psychometrics and screening accuracy properties of a chatbot system that understands free-text responses to mental health screening questions using natural language processing (NLP). The aiCARE system was tested against web-based versions of the Patient Health Questionnaire- 9 (PHQ-9), General Anxiety Disorder-7 (GAD-7), and Posttraumatic Stress Disorder Checklist (PCL-5). The study included 773 volunteers (Mage = 21.28, SD = 5.34) who answered the same free-text (chatbot version) and closed-ended survey questions (standard survey version). Overall, the research found that the proposed chatbot system is reliable in determining whether clinical symptomatology is present or absent based on free-text responses to PHQ-9, GAD-7, and PCL-5 questions. It had comparable sensitivity, specificity, total accuracy, and AUC values to standard web-based survey methods, as well as good internal consistency and convergent validity. The general implications are that chatbot systems could be used to identify common psychopathology as part of a stepped care model. It is not intended to be used in place of clinical diagnosis. Future research is needed to assess its effectiveness in more clinically and demographically diverse populations.
Read full abstract