Psychological disorder detection: A multimodal approach using a transformer-based hybrid model

Abstract

Detecting psychological disorders, particularly depression, is a complex and critical task within the realm of mental health assessment. This research explores a novel approach to improve the identification of psychological distresses, such as depression, by addressing the subjectivity, complexity, and biasness inherent in traditional diagnostic techniques. Using multimodal data, such as voice characteristics and linguistic content from participant interviews, we developed a Transformer-Based Hybrid Model that combines advanced natural language processing and deep learning approaches. This model provides a complete assessment of an individual's psychological well-being by merging aural cues and textual data. This study investigates the theoretical underpinnings, technical complexities, and practical applications of this model in the context of psychological disorder detection. Additionally, the model's design and implementation details are thoroughly documented to ensure replicability by other researchers.•A unique way of strengthening emotional ailments (focusing on depression).•Transformer-Based Hybrid Model is proposed using multimodal data from interviews of participants.•The model integrates voice characteristics (aural cues) and linguistic content (textual data).•Comparative analysis of this research with existing approaches.