The Presence of Bias Based on Pre-trained Language Models

Abstract

One of the most active fields of machine learning research is natural language processing (NLP). Although existing linguistic machine learning models excel numerically on a variety of linguistic comprehension tasks, they frequently lack implicit bias reduction optimization. To ensure that deep learning models can avoid the traps of implicit bias and that robots can make fair judgments, bias in NLP must be addressed adequately. The ramifications of permitting biased models to reach the actual world are serious. Thus must address this issue as quickly as feasible. This paper conducted data validation-bias experiments on several real datasets to verify the presence of gender bias in the pre-trained models, then proposed a word vector balancing algorithm to modify the actual vector representation of words by biasing the models and verifying the effectiveness of the debiasing method through debiasing experiments to mitigate the gender bias of the models while ensuring data accuracy, thereby improving the fairness of the classification results. Furthermore, this paper provides more accurate information for the future development and use of deep learning.