Spam classification based on parallel optimized BERT

Abstract

With the popularity of email and the increase in spam, effectively filtering spam has become an urgent need. This research presents a cutting-edge approach to spam filtering, leveraging the power of BERT and innovative parallel optimization techniques, improving email security. This study proposes a spam classification method based on the BERT (Bidirectional Encoder Representations from Transformers) model, aiming to improve the accuracy and efficiency of spam filtering. The study first investigated the shortcomings of traditional classification methods and then analysed the applicability of the BERT model to classifying spam. The study also performed two parallel optimizations on BERT, DDP (Distributed Data Parallel), and Gpipe. Among them, DDP was used to accelerate the large model BERT parallelly. After conducting experiments on spam classification, it was found that the average training time of an epoch was 54 seconds, with the accuracy on the test set reaching 98%, achieving significant performance improvements.