Real Time Multimodal Depression Analysis

Abstract

Severe mental disorder is recognized as Depression. State of low mood and aversion to activity causes abnormal behavior of a person in both professional and daily lives. As per WHO, around 350 million people worldwide are victimized by depression. Importance of automated real time mental health analysis is increasing day by day. In this paper, we proposed a system of automated depression diagnosis. This is a new approach to predict the depression severity corresponding to HAM-D score values obtained from prediction models. The proposed framework is designed keeping in mind a multi-modal approach, aiming at capturing facial characteristics, speech properties and brain waves. Further, a decision fusion technique has been implemented to integrate the obtained information in real-time. Using statistical features extracted from the speech recording, facial video and EEG data, the individual prediction models classify the subject according to severity of depression and the outputs are then fused to increase the performance parameters. The training data was obtained from 50 subjects, who provided all three recordings necessary for analysis. In unimodal systems the EEG data provides 80%, Speech 78% and Facial recording 72% accuracy, which is much inferior to a multimodal framework which provides 92% accuracy. The experimental results show that the proposed multimodal framework significantly improves the depression prediction performance, compared to other techniques. Inferior to a multimodal framework which provides 92% accuracy. The experimental results show that the proposed multimodal framework significantly improves the depression prediction performance, compared to other techniques.