A Transformer Based Approach To Detect Suicidal Ideation Using Pre-Trained Language Models

Abstract

Detection of Suicidal Ideation in social media has gained special attention in recent years. Different mental health issues like depression, frustration, hopelessness etc directly or indirectly influence suicidal thoughts. Early detection of suicidal thoughts can help people to diagnose and get proper treatment in time. Machine Learning and Deep Learning are playing a vital role in this area to automatize Suicidal Ideation detection. In our study, we wanted to take this trend to the next level and proposed a new detection approach with the help of the Transformer models in the language domain. Basically with Transformers we wanted to analyze raw social media posts and classify the indication of suicidal ideation in them. First, we applied BiLSTM(Bidirectional Long Short- Term Memory) and from there took a jump to Transformer models like BERT (Bidirectional Encoder Representations from Transformers), ALBERT, ROBERTa and XLNET - a complete robust Transformer model based study. The main advantage of Transformer models is having pre-trained language models. Because of this, these models usually perform better. We found they work far better than conventional Deep Learning architecture like Bi-LSTM in our work.