Attention-Enabled Ensemble Deep Learning Models and Their Validation for Depression Detection: A Domain Adoption Paradigm

Abstract

Depression is increasingly prevalent, leading to higher suicide risk. Depression detection and sentimental analysis of text inputs in cross-domain frameworks are challenging. Solo deep learning (SDL) and ensemble deep learning (EDL) models are not robust enough. Recently, attention mechanisms have been introduced in SDL. We hypothesize that attention-enabled EDL (aeEDL) architectures are superior compared to attention-not-enabled SDL (aneSDL) or aeSDL models. We designed EDL-based architectures with attention blocks to build eleven kinds of SDL model and five kinds of EDL model on four domain-specific datasets. We scientifically validated our models by comparing “seen” and “unseen” paradigms (SUP). We benchmarked our results against the SemEval (2016) sentimental dataset and established reliability tests. The mean increase in accuracy for EDL over their corresponding SDL components was 4.49%. Regarding the effect of attention block, the increase in the mean accuracy (AUC) of aeSDL over aneSDL was 2.58% (1.73%), and the increase in the mean accuracy (AUC) of aeEDL over aneEDL was 2.76% (2.80%). When comparing EDL vs. SDL for non-attention and attention, the mean aneEDL was greater than aneSDL by 4.82% (3.71%), and the mean aeEDL was greater than aeSDL by 5.06% (4.81%). For the benchmarking dataset (SemEval), the best-performing aeEDL model (ALBERT+BERT-BiLSTM) was superior to the best aeSDL (BERT-BiLSTM) model by 3.86%. Our scientific validation and robust design showed a difference of only 2.7% in SUP, thereby meeting the regulatory constraints. We validated all our hypotheses and further demonstrated that aeEDL is a very effective and generalized method for detecting symptoms of depression in cross-domain settings.