DPre: Effective preprocessing techniques for social media depressive text

Abstract

Depression has become one of the most common public health issues. Several people with depression rely on social media to express their grief. The text data generated by these users can be exploited to promote study in this area in order to detect early-stage depression and provide support. However, to develop a reliable automatic depression detection system, the social media text cannot be used directly as there is a lot of irrelevant, inaccurate, and noisy information available. Moreover, the basic preprocessing steps which are used with most of the machine learning models have limited functionality and thus lead to lots of information loss. This loss of information is not affordable especially in the domain of affective computing (mental health) for text. In this paper, we present various preprocessing techniques for depressive text, DPre, to obtain readable text from raw and noisy tweets. This method can help in minimizing the loss of information and expressions hidden in the raw tweet. Moreover, the processed and clean text will be ready to input into any machine learning algorithm. The readability of the processed text is evaluated and compared with raw tweets using four readability scores: Flesch Reading Score, Flesch_kincaid Score, the Coleman-Liau Index, and Dale_Chall Score. Compared to basic state-of-art preprocessing methods, the proposed method significantly improved the readability score.