Chinese Word Segmentation Model Research Articles

A Levenshtein distance-based method for word segmentation in corpus augmentation of geoscience texts

ABSTRACT For geoscience text, rich domain corpora have become the basis of improving the model performance in word segmentation. However, the lack of domain-specific corpus with annotation labelled has become a major obstacle to professional information mining in geoscience fields. In this paper, we propose a corpus augmentation method based on Levenshtein distance. According to the technique, a geoscience dictionary of 20,137 words was collected and constructed by crawling the keywords from published papers in China National Knowledge Infrastructure (CNKI). The dictionary was further used as the main source of synonyms to enrich the geoscience corpus according to the Levenshtein distance between words. Finally, a Chinese word segmentation model combining the BERT, Bi-gated recurrent neural network (Bi-GRU), and conditional random fields (CRF) was implemented. Geoscience corpus composed of complex long specific vocabularies has been selected to test the proposed word segmentation framework. CNN-LSTM, Bi-LSTM-CRF, and Bi-GRU-CRF models were all selected to evaluate the effects of Levenshtein data augmentation technique. Experiments results prove that the proposed methods achieve a significant performance improvement of more than 10%. It has great potential for natural languages processing tasks like named entity recognition and relation extraction.

A hybrid Chinese word segmentation model for quality management-related texts based on transfer learning.

Text information mining is a key step to data-driven automatic/semi-automatic quality management (QM). For Chinese texts, a word segmentation algorithm is necessary for pre-processing since there are no explicit marks to define word boundaries. Because of intrinsic characteristics of QM-related texts, word segmentation algorithms for normal Chinese texts cannot be directly applied. Hence, based on the analysis of QM-related texts, we summarized six features, and proposed a hybrid Chinese word segmentation model by means of integrating transfer learning (TL), bidirectional long-short term memory (Bi-LSTM), multi-head attention (MA), and conditional random field (CRF) to construct the mTL-Bi-LSTM-MA-CRF model, considering insufficient samples of QM-related texts and excessive cutting of idioms. The mTL-Bi-LSTM-MA-CRF model is composed of two steps. Firstly, based on a word embedding space, the Bi-LSTM is introduced for context information learning, and the MA mechanism is selected to allocate attention among subspaces, and then the CRF is used to learn label sequence constraints. Secondly, a modified TL method is put forward for text feature extraction, adaptive layer weights learning, and loss function correction for selective learning. Experimental results show that the proposed model can achieve good word segmentation results with only a relatively small set of samples.

GeoBERTSegmenter: Word Segmentation of Chinese Texts in the Geoscience Domain Using the Improved BERT Model

AbstractUnlike English, there is no natural separator‐like gap between words in Chinese, which makes Chinese word segmentation (CWS) a difficult information processing problem. At present, geological texts contain a large number of unregistered geological terms, and the existing rule‐based methods and machine‐learning and deep learning algorithms still cannot be used to solve the problem of word segmentation in geosciences, especially for the large number of unregistered words. In this study, we propose GeoBERTSegmenter, which is a GeoBERT‐based (Geoscience‐Bidirectional Encoder Representation from Transformers) CWS model that is specifically designed with various linguistic irregularities in mind. In this method, a general model is extended to a BERT bidirectional recurrent neural network (BiLSTM) and conditional random field (GeoBERT + BiLSTM + CRF) model with a number of features designed to address the CWS task in geological text. We also train a pretrained language model named GeoBERT on a geological domain that is based on a large amount of Chinese geological text. In open testing, a precision of 94.77%, recall of 96.31% and F1 of 95.44%, are obtained, indicating that the proposed strategy performs much better than alternative methods in our study. In this study, unregistered geological terms can be effectively identified, and the recognition rate of common words is ensured, which lays the foundation for natural language processing in the domain of geoscience through Chinese text word segmentation.

Natural Language Processing Using Neighbour Entropy-based Segmentation

In natural language processing (NLP) of Chinese hazard text collected in the process of hazard identification, Chinese word segmentation (CWS) is the first step to extracting meaningful information from such semi-structured Chinese texts. This paper proposes a new neighbor entropy-based segmentation (NES) model for CWS. The model considers the segmentation benefits of neighbor entropies, adopting the concept of "neighbor" in optimization research. It is defined by the benefit ratio of text segmentation, including benefits and losses of combining the segmentation unit with more information than other popular statistical models. In the experiments performed, together with the maximum-based segmentation algorithm, the NES model achieves a 99.3% precision, 98.7% recall, and 99.0% f-measure for text segmentation; these performances are higher than those of existing tools based on other seven popular statistical models. Results show that the NES model is a valid CWS, especially for text segmentation requirements necessitating longer-sized characters. The text corpus used comes from the Beijing Municipal Administration of Work Safety, which was recorded in the fourth quarter of 2018.

Multi-Task Joint Learning Model for Chinese Word Segmentation and Syndrome Differentiation in Traditional Chinese Medicine.

Evidence-based treatment is the basis of traditional Chinese medicine (TCM), and the accurate differentiation of syndromes is important for treatment in this context. The automatic differentiation of syndromes of unstructured medical records requires two important steps: Chinese word segmentation and text classification. Due to the ambiguity of the Chinese language and the peculiarities of syndrome differentiation, these tasks pose a daunting challenge. We use text classification to model syndrome differentiation for TCM, and use multi-task learning (MTL) and deep learning to accomplish the two challenging tasks of Chinese word segmentation and syndrome differentiation. Two classic deep neural networks—bidirectional long short-term memory (Bi-LSTM) and text-based convolutional neural networks (TextCNN)—are fused into MTL to simultaneously carry out these two tasks. We used our proposed method to conduct a large number of comparative experiments. The experimental comparisons showed that it was superior to other methods on both tasks. Our model yielded values of accuracy, specificity, and sensitivity of 0.93, 0.94, and 0.90, and 0.80, 0.82, and 0.78 on the Chinese word segmentation task and the syndrome differentiation task, respectively. Moreover, statistical analyses showed that the accuracies of the non-joint and joint models were both within the 95% confidence interval, with < 0.05. The experimental comparison showed that our method is superior to prevalent methods on both tasks. The work here can help modernize TCM through intelligent differentiation.

A Transformer-based Neural Model for Chinese Word Segmentation and Part-of-Speech Tagging

A Transformer-based Neural Model for Chinese Word Segmentation and Part-of-Speech Tagging

Domain Adaptive Chinese Word Segmentation Method for power data security classification

Chinese word segmentation (CWS) is an important task for Chinese NLP, and also an essential pre-processing step to establish a word-root database for security classification of power data, covering different domains such as laws & regulations, power. It is impracticable to label a large number of training corpus for each domain, which brings great challenge to the supervised statistical learning method to carry out effective CWS. Therefore, a Chinese word segmentation approach based on dictionary and semi-supervised conditional random field (SS-CRF ) is presented. At first, a CRF model for CWS is trained with self-training and active learning algorithms and used to conduct CWS task. Then the dictionary features are introduced to correct the result of CRF based segmentation by adopting RMM algorithm. Experiments on a cross domain segmentation task show that the proposed method can effectively improve the domain-adaptive performance of CWS.

Hybrid Feature Fusion Learning Towards Chinese Chemical Literature Word Segmentation

The rapid increase in the number of chemical science literature has brought challenges to researchers in search and data analysis. For many chemical scientific literature, extracting information from text and using knowledge is the focus of research. However, the existing Chinese text word segmentation methods have low recognition rate for chemical terms. The reason is that the addition of many new vocabulary and mixed professional vocabulary of Chinese and English brings challenges to word segmentation. In this paper, we propose a word segmentation method of Chinese chemical literature based on hybrid feature fusion learning Model(HFFLM). HFFLM first establishes chemical science corpus (Chem-pku) to train Chinese word segmentation (CWS) tasks. In addition, HFFLM uses BiLSTM and CNN to extract document features and fuse them. Then, HFFLM combines boundary features to construct conditional random field to train the end-to-end CWS model. In the end, HFFLM makes visual analysis of the word segmentation results. The experimental results indicate that HFFLM has high accuracy and recall rate, and is suitable for chemical industry vocabulary extraction with mixed Chinese and English.

An improved neural network for domain adaptive Chinese word segmentation

Various methods have been proposed recently to solve the problem of weak domain adaptability of Chinese word segmentation (CWS) models based on neural networks. However, although some of these improved models achieve high segmentation accuracy in a specific domain, they need to be retrained when applied to another. After rethinking the domain adaptability, two criteria, including the segmentation accuracy and the universality, are suggested for measuring it. Taking the above two criteria into consideration, an improved neural-based CWS model is proposed, which incorporates the common lexicon and unlabeled data into BERT. To make the most use of lexicon, a new method is proposed to construct the lexicon-based feature vector. In addition, the domain-specific words can be effectively extracted by pre-training a language model on the unlabeled data. Finally, a GRU-like gate structure is used to integrate the lexicon-based feature vector and language model into BERT. Experiments on five different domains reveal that the domain adaptability of this model is very strong.

Capsules Based Chinese Word Segmentation for Ancient Chinese Medical Books

Neural network models are popularly used in Chinese word segmentation task. The capsule architecture is proposed recently which has solved some defects of convolutional neural network. In this paper, we first introduce the capsule architecture to Chinese word segmentation. We utilize capsules as neural units. Before doing routing algorithm, we make a sliding capsule window to select the features which are extracted from the primary capsule layer. The sliding capsule window is proposed to adapt the capsule architecture to the sequence labeling task. The experiment results show that our proposed capsules based Chinese word segmentation model achieves competitive performances with the previous state-of-the-art methods. Ancient Chinese medical books record a lot of valuable experiences from the ancient medical workers. However, the research about the automatic text analysis on ancient Chinese medical documents is just a beginning. Due to the lack of the annotated data for Chinese medicine, we develop the word segmentation guideline for the ancient Chinese medical documents and select 10 genres, 30 ancient Chinese medical books to set up the annotation dataset. And with the annotated data, we develop the segmenter for the ancient Chinese medical text. Experiments show that the $F_{1}$ measures of our model on the two datasets are 94.9% and 81.4% on Chinese Treebank6.0 and Ancient Chinese Medical Books, respectively.

Method of Chinese word segmentation based on character-word classification

Chinese Word Segmentation CWS is the basic problem of natural language processing.A joint characterword classification model for Chinese Word Segmentation was presented which mainly dealt with the problem of Conditional Random Field CRF.On the one hand the majority of errors in CRF caused by fine granularity were figured out in outsidelayer of the model.On the other hand excessive linked words caused by improper segmentations that were settled in core inside-layer.The experimental results show that the value of F is greatly improved and the good results of word segmentation are easily gained by choosing the hierarchical word segmentation model based on character word classification.

Punctuation as Implicit Annotations for Chinese Word Segmentation

We present a Chinese word segmentation model learned from punctuation marks which are perfect word delimiters. The learning is aided by a manually segmented corpus. Our method is considerably more effective than previous methods in unknown word recognition. This is a step toward addressing one of the toughest problems in Chinese word segmentation.

A Hybrid Model for Chinese Word Segmentation

This paper describes a hybrid model that combines machine learning with linguistic and statistical heuristics for integrating unknown word identification with Chinese word segmentation. The model consists of two major components: a tagging component that annotates each character in a Chinese sentence with a position-of-character (POC) tag that indicates its position in a word, and a merging component that transforms a POC-tagged character sequence into a word-segmented sentence. The tagging component uses a support vector machine (Vapnik, 1995) based tagger to produce an initial tagging of the text and a transformation-based tagger (Brill, 1995) to improve the initial tagging. In addition to the POC tags assigned to the characters, the merging component incorporates a number of linguistic and statistical heuristics to detect words with regular internal structures, recognize long words, and filter non-words. Experiments show that, without resorting to a separate unknown word identification mechanism, the model achieves an F-score of 95.0% for word segmentation and a competitive recall of 74.8% for unknown word identification.