MIATS:A Chinese Spelling Error Correction Algorithm Based on Multimodal Information Alignment of Three-Towers Structure

Abstract

Chinese spelling correction (CSC) is a crucial task in natural language processing, aiming to detect and correct spelling errors in Chinese text. The improved performance of Chinese spelling errors correction algorithms can enhance the efficiency and accuracy of upstream and downstream Chinese natural language processing tasks, such as OCR, ASR, and translation.However, current methods based on neural networks are mostly limited to either using only contextual information to correct misspelled words or failing to fully utilize glyph and pinyin information. Therefore, we propose a multimodal approach to address the above issues. Specifically, a three-tower multimodal structure is used to extract glyph, pinyin, and semantic information, and a decoder composing of an error probability prediction network and a transformer network is employed to achieve cross-modal information interaction. Besides, an additional training task is used to achieve cross-modal information alignment. Experiments demonstrate that proposed network outperform most existing motheds.