Clinical short text classification method based on ALBERT and GAT

Abstract

As one of the key tasks in natural language processing, the accuracy of short text classification will directly affect the performance of subsequent downstream tasks. Most of the existing short text classification algorithms usually do not understand the deep semantic information related to the medical field and lack the feature extraction of medical proprietary vocabulary. Traditional models often extract features from context information, which is limited by the lack of access to global and deep semantic information. This paper presents a short text classification algorithm for Chinese clinical medicine combining ALBERT pre-training model and graph attention network. The ALBERT pre-training model is used to represent the feature vectors of the short text of the problem, and then the graphical structure data is generated from the feature vectors. GAT is used to assign different weights to each node in a uniform neighborhood. Finally, a map-level semantic representation for category prediction is generated. The experimental results show that the accuracy of the model is 83.27% on Chinese clinical problem datasets, which effectively improves the model performance.