Retraction to: M111. QUANTITATIVE ASSESSMENT OF MANIA AND PSYCHOSIS DURING HOSPITALIZATION USING AUTOMATED ANALYSIS OF FACE, VOICE, AND LANGUAGE

Abstract

Abstract Background A major challenge for reliable and effective mental health care is the lack of objective markers of illness. Computational approaches to measuring naturalistic behavior in clinical settings could therefore provide an objective backstop for mental health assessment and disease monitoring. This study aimed to train machine-learning (ML) classifiers to estimate conventional clinical measures of severe mental illness using quantitative metrics derived from computational analysis of facial and vocal behaviors. Methods Individuals hospitalized for any active psychotic condition were recruited to participate in up to ten recorded study visits, comprised of three segments. Each visit was captured using two synchronized HD webcams and cardioid microphones, to obtain high quality audiovisual (AV) data from both patient and interviewer. We performed automated facial action coding, vocal analysis, and speech transcription using publicly available software (e.g., openFace, openSmile, TranscribeMe). Results A total of 34 participants, participated in 66 sessions between 2015 and 2018, resulting in over 40 hours of AV recordings. In our visual and vocal analysis, we found that several features derived from face, voice, and use of language (i.e. eyebrow furrowing, eye widening, smile variability, characteristics of vowels) were both robustly measured using our approach, and allowed us to accurately estimate multiple symptom domains (i.e. mania, depression, psychosis) with (R= >0.7, p = <0.05). In our linguistic analysis, we found that abundance of power words (i.e. superiority, important) and lack of contextual language (i.e. yesterday, nearby) are highly indicative of positive psychotic symptoms with (R= +0.417, p = 0.002) and (R= -0.302, p = 0.028) respectively. Discussion Automated analysis of face, voice, and speech provides a number of robust behavioral markers sensitive enough to detect changes in psychopathology within individuals over time. Therefore, naturalistic, quantitative assessments can yield objective markers of mood and cognition that can be used to optimize both access and quality of treatments for a wide range of psychiatric conditions.