Multi-Task Fine-Tuning on BERT Using Spelling Errors Correction for Chinese Text Classification Robustness

Abstract

Spelling errors are common in our daily life and the industrial application, caused by automatic speech recognition, optional character recognition and human writing. Because of lack of robustness, Text classification models trained on clean datasets tend to perform poorly on the datasets with spelling errors. We conduct experiments to find out the influence of spelling errors on the performance of Chinese text classification and solve the Chinese text classification task with spelling errors by multi-task fine-tuning on BERT. We use spelling errors correction task to assist the text classification task. The results on four Chinese text classification datasets show that our method can effectively improve the robustness of the classification model which decrease the influence of spelling errors and prove the effectiveness of multi-task fine-tuning on BERT.