THFN: Emotional health recognition of elderly people using a Two-Step Hybrid feature fusion network along with Monte-Carlo dropout

Abstract

The influence of technology’s progress on the ability to read people's emotions has received increased attention in recent years. It is essential to identify the elderly's feelings because it indicates mental wellness. This study proposes a unique approach to micro-information extraction using a cross-model attention mechanism and a two-step hybrid feature fusion Network (THFN). To predict feelings across many channels, the study suggests using a cross-model attention network. The model improves the accuracy of emotion prediction in the elderly by employing a fusion mechanism at both the feature level and the decision level. In addition, Monte-Carlo dropout (MCD) in the deep neural model is employed for improved classification and uncertainty prediction. As an activation function, the first-order derivative of Mish (Fmish) helps to boost classification accuracy. ElderReact is a brand-new multimodal dataset which is employed for senior citizens' responses to stimuli designed to evoke emotions. This research contributes to the emotional lives of the elderly, which helps to design better involvements to improve their mental health. The experimental results show that the proposed approach achieves a precision of 0.99, Recall of 0.99, F1-score of 1.00 and 99.8% accuracy, the sensitivity of 99.7% and specificity of 98.9% in emotion classification, and 100% accuracy in emotional health prediction. From the experimental results, it is proved that the proposed model performs better than the previously suggested models.