Chinese NLP Research Articles

Predicting Chinese tourists’ B&B preferences through a method of online reviews causality analytic

With the enormous numbers of online reviews posted daily, automatically acquired and analyzed the information that rich of causal logics is of great significance for Chinese B&B enterprises to make accurate business decisions. This study proposed a method based on Chinese NLP technologies to reveal and describe the formation process of tourists' behavioral preferences better than the previous entity relationship extraction and analysis methods. Firstly, Cause-effect syntactic and template pattern had been applied to extraction event casual relations in the online reviews texts. Then, Doc2vec-Kmeans clustering algorithm and key-phrases sorting algorithm were used to generalize data. Finally, causality graph was constructed for visual analysis. To evaluate the performance of the proposed approach, we collected 549, 732 Chinese B&B reviews from Ctrip.com and Qunaer.com. The survey identified 7 common preferences that travelers' concern about nowadays and reveals the reasons behind each of them, as well as concluded that the ‘services’ and ‘conveniences’ of Chinese B&Bs are the trends of travelers’ preferences in the future selections. The results indicated that the proposed approach outperforms the other entity relation extraction method in event forecasting, and most of the subjects believed that the proposed approach can provide more comprehensive and logical traveler's potential demand information.

Chinese Idiom Paraphrasing

AbstractIdioms are a kind of idiomatic expression in Chinese, most of which consist of four Chinese characters. Due to the properties of non-compositionality and metaphorical meaning, Chinese idioms are hard to be understood by children and non-native speakers. This study proposes a novel task, denoted as Chinese Idiom Paraphrasing (CIP). CIP aims to rephrase idiom-containing sentences to non-idiomatic ones under the premise of preserving the original sentence’s meaning. Since the sentences without idioms are more easily handled by Chinese NLP systems, CIP can be used to pre-process Chinese datasets, thereby facilitating and improving the performance of Chinese NLP tasks, e.g., machine translation systems, Chinese idiom cloze, and Chinese idiom embeddings. In this study, we can treat the CIP task as a special paraphrase generation task. To circumvent difficulties in acquiring annotations, we first establish a large-scale CIP dataset based on human and machine collaboration, which consists of 115,529 sentence pairs. In addition to three sequence-to-sequence methods as the baselines, we further propose a novel infill-based approach based on text infilling. The results show that the proposed method has better performance than the baselines based on the established CIP dataset.

SWSR: A Chinese dataset and lexicon for online sexism detection

Online sexism has become an increasing concern in social media platforms as it has affected the healthy development of the Internet and can have negative effects in society. While research in the sexism detection domain is growing, most of this research focuses on English as the language and on Twitter as the platform. Our objective here is to broaden the scope of this research by considering the Chinese language on Sina Weibo. We propose the first Chinese sexism dataset – Sina Weibo Sexism Review (SWSR) dataset –, as well as a large Chinese lexicon SexHateLex made of abusive and gender-related terms. We introduce our data collection and annotation process, and provide an exploratory analysis of the dataset characteristics to validate its quality and to show how sexism is manifested in Chinese. The SWSR dataset provides labels at different levels of granularity including (i) sexism or non-sexism, (ii) sexism category and (iii) target type, which can be exploited, among others, for building computational methods to identify and investigate finer-grained gender-related abusive language. We conduct experiments for the three sexism classification tasks making use of state-of-the-art machine learning models. Our results show competitive performance, providing a benchmark for sexism detection in the Chinese language, as well as an error analysis highlighting open challenges needing more research in Chinese NLP. The SWSR dataset and SexHateLex lexicon are publicly available.11https://doi.org/10.5281/zenodo.4773875.

Domain Adaptive Chinese Word Segmentation Method for power data security classification

Chinese word segmentation (CWS) is an important task for Chinese NLP, and also an essential pre-processing step to establish a word-root database for security classification of power data, covering different domains such as laws & regulations, power. It is impracticable to label a large number of training corpus for each domain, which brings great challenge to the supervised statistical learning method to carry out effective CWS. Therefore, a Chinese word segmentation approach based on dictionary and semi-supervised conditional random field (SS-CRF ) is presented. At first, a CRF model for CWS is trained with self-training and active learning algorithms and used to conduct CWS task. Then the dictionary features are introduced to correct the result of CRF based segmentation by adopting RMM algorithm. Experiments on a cross domain segmentation task show that the proposed method can effectively improve the domain-adaptive performance of CWS.

Pre-Training With Whole Word Masking for Chinese BERT

Bidirectional Encoder Representations from Transformers (BERT) has shown marvelous improvements across various NLP tasks, and its consecutive variants have been proposed to further improve the performance of the pre-trained language models. In this paper, we aim to first introduce the whole word masking (wwm) strategy for Chinese BERT, along with a series of Chinese pre-trained language models. Then we also propose a simple but effective model called MacBERT, which improves upon RoBERTa in several ways. Especially, we propose a new masking strategy called MLM as correction (Mac). To demonstrate the effectiveness of these models, we create a series of Chinese pre-trained language models as our baselines, including BERT, RoBERTa, ELECTRA, RBT, etc. We carried out extensive experiments on ten Chinese NLP tasks to evaluate the created Chinese pre-trained language models as well as the proposed MacBERT. Experimental results show that MacBERT could achieve state-of-the-art performances on many NLP tasks, and we also ablate details with several findings that may help future research. We open-source our pre-trained language models for further facilitating our research community.

CPM: A large-scale generative Chinese Pre-trained language model

Pre-trained Language Models (PLMs) have proven to be beneficial for various downstream NLP tasks. Recently, GPT-3, with 175 billion parameters and 570 GB training data, drew a lot of attention due to the capacity of few-shot (even zero-shot) learning. However, applying GPT-3 to address Chinese NLP tasks is still challenging, as the training corpus of GPT-3 is primarily English, and the parameters are not publicly available. In this technical report, we release the Chinese Pre-trained Language Model (CPM) with generative pre-training on large-scale Chinese training data. To the best of our knowledge, CPM, with 2.6 billion parameters and 100 GB Chinese training data, is the largest Chinese pre-trained language model, which could facilitate several downstream Chinese NLP tasks, such as conversation, essay generation, cloze test, and language understanding. Extensive experiments demonstrate that CPM achieves strong performance on many NLP tasks in the settings of few-shot (even zero-shot) learning. The code and parameters are available at https://github.com/TsinghuaAI/CPM.

An Algorithm Based on Simple CNN and BI_LSTM Network for Chinese Word Segmentation

In dealing with most Chinese NLP tasks, word segmentation is an indispensable and critical work, and it also affects the accuracy of subsequent tasks. Neither the traditional BI_LSTM network can extract features effectively, nor the CNN network can deal with the timing problem effectively. As a result, this paper proposes a new algorithm to solve these problems. First, the Chinese characters’ feature in a whole sentence is extracted and recombined using the CNN network; then, the recombined feature is combined in BI_LSTM network; finally, each word of the whole sentence which classified by weight sharing and the corresponding classification of each word is the consequent output. Not only the advantages of CNN network are utilized for feature extraction, but also the advantages of BI_LSTM network are retained for timing processing. In conclusion, the learning ability of BI_LSTM network is increased, and the accuracy of output is improved to 98%.

Towards Chinese clinical named entity recognition by dynamic embedding using domain-specific knowledge

The task of electronic medical record named entity recognition (NER) refers to automatically identify all kinds of named entities in the medical record text. Chinese clinical NER remains a major challenge. One of the main reasons is that Chinese word segmentation will lead to the wrong downstream works. Besides, existing methods only use the information of the general field, not consider the knowledge from field of medicine. To address these issues, we propose a dynamic embedding method based on dynamic attention which combines features of both character and word in embedding layer. Domain knowledge is provided by word vector trained by domain dataset. In addition, spatial attention is added to enable the model to obtain more and more effective context encoding information. Finally, we conduct extensive experiments to demonstrate the effectiveness of our proposed algorithm. Experiments on CCKS2017 and Common dataset shows that the proposed method outperforms the baseline.

Lattice LSTM for Chinese Sentence Representation

Words provide a useful source of information for Chinese NLP, and word segmentation has been taken as a pre-processing step for most downstream tasks. For many NLP tasks, however, word segmentation can introduce noise and lead to error propagation. The rise of neural representation learning models allows sentence-level semantic information to be collected from characters directly. As a result, it is an empirical question whether a fully character-based model should be used instead of first performing word segmentation. We investigate a neural representation that simultaneously encodes character and word information without the need for segmentation. In particular, candidate words are found in a sentence by matching with a pre-defined lexicon. A lattice structured LSTM is used to encode the resulting word-character lattice, where gate vectors are used to control information flow through words, so that the more useful words can be automatically identified by end-to-end training. We compare the performance of the resulting lattice LSTM and baseline sequence LSTM structures over both character sequences and automatically segmented word sequences. Results on NER show that the character-word lattice model can significantly improve the performance. In addition, as a general sentence representation architecture, character-word lattice LSTM can also be used for learning contextualized representations. To this end, we compare lattice LSTM structure with its sequential LSTM counterpart, namely ELMo. Results show that our lattice version of ELMo gives better language modeling performances. On Chinese POS-tagging, chunking and syntactic parsing tasks, the resulting contextualized Chinese embeddings also give better performance than ELMo trained on the same data.

Neural Chinese word segmentation with dictionary

Chinese word segmentation (CWS) is an important task for Chinese NLP. Recently, many neural network based methods have been proposed for Chinese word segmentation. However, these methods require a large number of labeled sentences for model training, and usually cannot utilize the useful information in Chinese dictionary. In this paper, we propose two methods to exploit the dictionary information for CWS. The first one is based on pseudo labeled data generation, and the second one is based on multi-task learning. The experimental results on two benchmark datasets validate that our approach can effectively improve the performance of Chinese word segmentation, especially when training data is insufficient.

Word intuition agreement among Chinese speakers: a Mechanical Turk-based study

Word intuition is speakers’ intuitive knowledge on wordhood. Collective word intuition is the word intuition of the whole language community. Given this definition, the optimal word segmentation result in Chinese NLP should reflect collective word intuition. It is also believed that an ideal definition of Chinese word should accord with the collective word intuition of Chinese speakers. To test the validity and feasibility of modeling collective word intuition, it is important to know to what extent Chinese speakers agree with each other on what is a word. In this study, we measured word intuition agreement using Mechanical Turk-based Chinese word segmentation experiment. Three metrics were used: proportionate agreement, Cohen’s kappa, and Fleiss’ kappa. The results show that Chinese speakers agree with each other almost perfectly on what is a word. And we found no evidence to support an effect of semantic transparency on word intuition agreement. Such high word intuition agreement among Chinese speakers supports the psychological reality of Chinese word and also suggests that that it is quite feasible to formulate a definition of Chinese word by modeling the collective word intuition of Chinese speakers.

An approach on Chinese microblog entity linking combining baidu encyclopaedia and word2vec

Microblog such as Twitter and Sina Weibo provides a convenient and instant platform which makes information easy to share and acquire. However, Microblog’s short, noisy, real-time features make Chinese Microblog entity linking task a new challenge. In this paper, we investigate many linking methods and introduce the implementation of our work on Chinese microblog entity linking task. By means of crawling Baidu encyclopaedia web page, we generate polysemous, synonymous and index collections in MongoDB to manage the entities. We use a Chinese NLP tools named HanLP1 to perform noun words extracting, and then generate candidate set with these collections and word similarity. For disambiguation part, we take Word2vec2 whose model is trained by THUC news3 to determine the textual relevance. Our work performs pretty well on the Sina Weibo data set.