Sentiment Classification on tweets Based on LSTM with variants and BERT finetune

Abstract

Covid-19 is a new type of epidemic, we performed sentiment classification tasks on Covid-19 tweets using different machine learning models. The famous pre-training models are not trained with the text relevant to COVID-19, a new kind of virus appearing at December 2019. The twitter posts with such a topic also have not been applied to test the performance of existing pre-training models and neural networks well. In our experiment, we used LSTM and Transformer to do the sentiment analysis (quinary classification) with a dataset including those twitter posts and tried different hyperparameters and models to improve the performance of classification. We finally found that the Transformer performs better than LSTM with an extra softmax layer in the encoder part, and the bidirectional transformer with 4 hidden layer and dropout 0.2 provides the best results among all hyperparameters we have tested. With the finetune of BERT,we got the best performance with the accuracy over 85%.