A new Classifier Chain method of BERT Models For Multi-label Classification of Arabic Abusive Language on Social Media

Abstract

Moving beyond classical single-label classification tasks proves to be crucial nowadays since the majority of real-life problems are inherently multi-label. Various tasks require involving, for one single instance as an input, several classes simultaneously or none of them. Particularly, detecting abusive texts in Arabic social networks intuitively carries several manifestations with regard to the targets’ referred characteristics like gender, race, sexual orientation, etc. Existing multi-label algorithms either invest considerable complexity to handle labels’ correlation or explicitly ignore this information to deal with each label as an independent binary problem. To remedy the abovementioned problems, this paper proposes a new multi-label classification solution based on deep neural networks. The main idea is to exploit the correlation between all the dataset's labels to break it into ordered subsets. Then, follow that order to implement a chain of BERT classifiers. Each classifier is therefore trained on a corresponding subset and passes its output via feature space to the following one. For a new instance, we gather all the classifiers’ results and pick the best one for each class. Experiments showed that our approach significantly outperformed the single classifier's method using BERT as well as other neural network architectures on our Arabic abusive language dataset. 91%,76%, and 82% were achieved respectively for precision, recall, and F1-score. Moreover, we further investigated the performance of our approach on Kaggle multi-label research paper dataset and achieved very promising results.