Word Segmentation Results Research Articles

Cross-Modal Impact of Recent Word Encountering Experience

ABSTRACT Purpose This study was designed to distinguish the degree of sharing of representations between different modalities by investigating whether a word encountering experience in one modality impacts word processing in another modality. Method In three experiments, participants experienced some words frequently in the auditory modality (Experiment 1, sample size 30, mean age 23.4, 56.7% female, all participants were native Chinese speakers) or visual modality (Experiment 2, sample size 30, mean age 22.4 years, 63.3% female, all participants were native Chinese speakers) in the training session, and were tested whether the word encountering experience impacts the results of Chinese word segmentation in the visual modality in the test session. In Experiment 3 (sample size 30, mean age 24.6 years, 76.7% female, all participants were native Chinese speakers), we used a within-subjects design, in which each participant received both auditory and visual training tasks. Results The results of the three experiments showed that encountering a word frequently in a short period of time in the auditory modality or visual modality can affect word segmentation in Chinese reading, with a recently experienced word being more likely to be segmented as a word. This effect was long-lasting, as it could still be observed after 7 days. Conclusion The results suggest that the effect of a word encountering experience in listening can be transferred to reading. Thus, word encountering experiences should be stored at a location in the mind that is used for both listening and reading.

Business Brand Research Based on Multi-Feature Fusion Emotion Analysis.

With the deepening of globalization, brand plays an important role in determining the competitiveness of enterprises. It is worth thinking about how to quantify the brand value reasonably to achieve the purpose of improving the competitiveness of enterprises. The research of commercial brands based on emotion analysis extracts the views of consumers on the evaluation data of brand attributes, analyzes the emotional tendency of consumers' views, and then helps enterprises adjust their production strategies. The purpose of emotion analysis is to judge users' views and attitudes toward goods or services by analyzing texts. In this paper, the linear support vector machine model is used to extract and study the features in sentiment analysis. Then, the brand based on the fusion model is evaluated, and the experimental conclusions are drawn: the method of fusing the depth features extracted by word vector and the named entity extracted by CRF makes the brand effect of fusing emotional factors better than the feature extraction method only using CRF named entity recognition. It is proved that the model method proposed in this paper has a certain role in actual business. Feature fusion often combines shallow model methods and fuses various shallow feature screening technologies based on word segmentation results. This paper introduces fusion features and supplements feature vectors based on the results of the deep learning word vector model. Support Vector Machine (SVM) maps feature vectors to some sample points in the space, finds a plane that can realize sample classification, and maximizes the distance between the two sample points closest to the plane in two types of samples, to maximize the classification performance and improve the generalization of the model.

A Feature-Rich Vietnamese Named Entity Recognition Model

In this paper, we present a feature-based named entity recognition (NER) model that achieves the start-of-the-art accuracy for Vietnamese language. We combine word, word-shape features, PoS, chunk, Brown-cluster-based features, and word-embedding-based features in the Conditional Random Fields (CRF) model. We also explore the effects of word segmentation, PoS tagging, and chunking results of many popular Vietnamese NLP toolkits on the accuracy of the proposed feature-based NER model. Up to now, our work is the first work that systematically performs an extrinsic evaluation of basic Vietnamese NLP toolkits on the downstream NER task. Experimental results show that while automatically-generated word segmentation is useful, PoS and chunking information generated by Vietnamese NLP tools does not show their benefits for the proposed feature-based NER model.

The Application of Unsupervised Learning TF-IDF Algorithm in Word Segmentation of Ideological and Political Education

Word segmentation is an essential part of Chinese learning. It often plays a decisive role in the quality of language processing results. Its domain vocabulary has the advantages of fast birth of new words and wide coverage. Large vocabulary, which has caused great difficulties in word segmentation and follow-up work in the field. This paper designs and implements a system. The system builds a corpus based on domain literature, trains a word-level language model based on statistical ideas, and uses the Viterbi algorithm to obtain preliminary Chinese word segmentation results. The word optimization algorithm optimizes the preliminary. The system provides users with functions such as keyword extraction, word frequency statistics, and word cloud drawing for the word segmentation results, so as to realize Chinese word segmentation and text analysis of field documents. The experimental results show that building system based on the corresponding ideas has improved the text analysis efficiency, related research on documents and text processing in the field have a certain promotion effect.

Design and application of computer conference+Thai corpus-oriented word segmentation consistency checking system

The construction of deep processing corpus is one of the hot topics in corpus linguistics and computational linguistics, and the quality of corpus directly affects the subsequent research based on this corpus. It is a very effective way to build a corpus of questions in restricted areas to help understand users’ questions. In view of the inconsistent results of word segmentation and part-of-speech tagging in Thai corpus, this paper puts forward a proofreading method and strategy based on computer conference, and designs a consistent proofreading system. This method extracts grammatical and semantic collocation information between words, judges candidate sequences by using the theory of support vector machine, and gives a segmentation result, thus improving the segmentation accuracy of Chinese corpus.

Digital Mining Algorithm of English Translation Course Information Based on Digital Twin Technology

Cross‐language communication puts forward higher requirements for information mining in English translation course. Aiming at the problem that the frequent patterns in the current digital mining algorithms produce a large number of patterns and rules, with a long execution time, this paper proposes a digital mining algorithm for English translation course information based on digital twin technology. According to the results of word segmentation and tagging, the feature words of English translation text are extracted, and the cross‐language mapping of text is established by using digital twin technology. The estimated probability of text translation is maximized by corresponding relationship. The text information is transformed into text vector, the semantic similarity of text is calculated, and the degree of translation matching is judged. Based on this data dimension, the frequent sequence is constructed by transforming suffix sequence into prefix sequence, and the digital mining algorithm is designed. The results of example analysis show that the execution time of digital mining algorithm based on digital twin technology is significantly shorter than that based on Apriori and Map Reduce, and the mining accuracy rate reached more than 80%, which has good performance in processing massive data.

Hybrid Feature Fusion Learning Towards Chinese Chemical Literature Word Segmentation

The rapid increase in the number of chemical science literature has brought challenges to researchers in search and data analysis. For many chemical scientific literature, extracting information from text and using knowledge is the focus of research. However, the existing Chinese text word segmentation methods have low recognition rate for chemical terms. The reason is that the addition of many new vocabulary and mixed professional vocabulary of Chinese and English brings challenges to word segmentation. In this paper, we propose a word segmentation method of Chinese chemical literature based on hybrid feature fusion learning Model(HFFLM). HFFLM first establishes chemical science corpus (Chem-pku) to train Chinese word segmentation (CWS) tasks. In addition, HFFLM uses BiLSTM and CNN to extract document features and fuse them. Then, HFFLM combines boundary features to construct conditional random field to train the end-to-end CWS model. In the end, HFFLM makes visual analysis of the word segmentation results. The experimental results indicate that HFFLM has high accuracy and recall rate, and is suitable for chemical industry vocabulary extraction with mixed Chinese and English.

Research on power grid scheduling log word vector extraction based on bidirectional LSTM combined dictionary

With the rapid development of China’s economy, the power network specifications are expanding and the network structure is becoming more and more complex. Power grid dispatching is the key to ensure the safe and stable operation of power grid. Power grid dispatch log is an important data source to reflect the operation of power grid and an important means to monitor the daily operation of power grid. Network dispatching log classification is an important application of log text analysis and mining. At present, there are many methods for network dispatching log classification, including naive bayesian method, support vector machine, neural network model and so on. However, no matter what classification method is used, scheduling log text needs to be preprocessed and converted into vector form before model training and classification. At present, the research of word vector mainly focuses on the Internet, while the feature extraction of power grid dispatch log from word vector generation is less. In this paper, a method of extracting log word vectors from power grid dispatching based on bidirectional LSTM combined dictionary is proposed. Firstly, the original log is preprocessed according to the lexicon, and word segmentation is performed on the original log by means of bidirectional LSTM combined with dictionary to obtain word segmentation results. Then, every word is transformed into a word vector through the skip-gram model. Finally, the generated word vector is used to classify the power grid dispatch logs.

Domain-Specific Chinese Word Segmentation Based on Bi-Directional Long-Short Term Memory Model

Most of the current word segmentation methods are rule-based and traditional machine learning methods. Universal word segmentation tools do not work well in the field such as metallurgy. Domain-specific Chinese word segmentation is rarely studied. In recent years, with the development of deep learning, the neural network has been proved to be effective in Chinese word segmentation. However, this promising performance relies on large-scale training data. Neural networks with conventional architectures cannot achieve the desired results in low-resource datasets due to the lack of labeled training data. This paper takes the field of metallurgy as an example and proposes a domain-specific Chinese word segmentation based on Bi-directional long-short term memory (Bi-directional LSTM) model in the metallurgical field. First, the word segmentation model is obtained by using the Bi-directional LSTM model to train the internal and external domain knowledge. Then, a series of tuning parameters are carried out and the label probability of the word is combined with the weight. Finally, the result of word segmentation is obtained by label inference layer. The experimental results show that the proposed method can create a better word segmentation effect in the field of metallurgy.

Chinese NER Using Dynamic Meta-Embeddings

Named entity recognition (NER) is one of the basic techniques in natural language processing tasks. Chinese NER is complicated and difficult which remains a major challenge. One of the main reasons is that the boundaries of entities in Chinese are blurred and closely related to word segmentation results. Previous studies for this task have broadly divided into two categories, word-based, and character-based methods. However, the former class suffers from the word segmentation errors, and the latter cannot make full use of the information on multiple granularities. To address these problems, we investigate a new dynamic meta-embeddings method and apply it to Chinese NER task, which utilizes attention mechanism to combine features of both character and word granularity in embedding layer. The meta-embeddings created by our method are dynamic, data-specific, and task-specific, since the meta-embeddings for same characters in different sentence sequences are distinct. The experiments on MSRA and LiteratureNER datasets validate the effectiveness of our model, and this method achieves the state-of-the-art results on LiteratureNER.

A Chinese Message Sensitive Words Filtering System based on DFA and Word2vec

In this paper, a Chinese message sensitive words filtering system applied in an instant messaging environment is proposed. Firstly, the message sentence is segmented, and the segmentation result is corrected by using the association algorithm based on information entropy and point mutual information. The traditional DFA algorithm is used to construct the dictionary tree for sensitive word recognition, which effectively improves the recognition speed. Secondly, on the basis of the completion of the recognition, the pre-trained word vector model is used to match the words in the sensitive words list and the word segmentation results, and the words with higher similarity with the sensitive words are added to the sensitive words list to achieve the expansion and improvement of the sensitive thesaurus.

Unsupervised spatial lexical acquisition by updating a language model with place clues

Abstract This paper describes how to achieve highly accurate unsupervised spatial lexical acquisition from speech-recognition results including phoneme recognition errors. In most research into lexical acquisition, the robot has no pre-existing lexical knowledge. The robot acquires sequences of some phonemes as words from continuous speech signals. In a previous study, we proposed a nonparametric Bayesian spatial concept acquisition method (SpCoA) that integrates the robot’s position and words obtained by unsupervised word segmentation from uncertain syllable recognition results. However, SpCoA has a very critical problem to be solved in lexical acquisition; the boundaries of word segmentation are incorrect in many cases because of many phoneme recognition errors. Therefore, we propose an unsupervised machine learning method (SpCoA++) for the robust lexical acquisition of novel words relating to places visited by the robot. The proposed SpCoA++ method performs an iterative estimation of learning spatial concepts and updating a language model using place information. SpCoA++ can select a candidate including many words that better represent places from multiple word-segmentation results by maximizing the mutual information between segmented words and spatial concepts. The experimental results demonstrate a significant improvement of the phoneme accuracy rate of learned words relating to place in the proposed method by word-segmentation results based on place information, in comparison to the conventional methods. We indicate that the proposed method enables the robot to acquire words from speech signals more accurately, and improves the estimation accuracy of the spatial concepts.

Chinese to Braille translation based on Braille word segmentation using statistical model

Automatic translation of Chinese text to Chinese Braille is important for blind people in China to acquire information using computers or smart phones. In this paper, a novel scheme of Chinese-Braille translation is proposed. Under the scheme, a Braille word segmentation model based on statistical machine learning is trained on a Braille corpus, and Braille word segmentation is carried out using the statistical model directly without the stage of Chinese word segmentation. This method avoids establishing rules concerning syntactic and semantic information and uses statistical model to learn the rules stealthily and automatically. To further improve the performance, an algorithm of fusing the results of Chinese word segmentation and Braille word segmentation is also proposed. Our results show that the proposed method achieves accuracy of 92.81% for Braille word segmentation and considerably outperforms current approaches using the segmentation-merging scheme.

Method of Chinese word segmentation based on character-word classification

Chinese Word Segmentation CWS is the basic problem of natural language processing.A joint characterword classification model for Chinese Word Segmentation was presented which mainly dealt with the problem of Conditional Random Field CRF.On the one hand the majority of errors in CRF caused by fine granularity were figured out in outsidelayer of the model.On the other hand excessive linked words caused by improper segmentations that were settled in core inside-layer.The experimental results show that the value of F is greatly improved and the good results of word segmentation are easily gained by choosing the hierarchical word segmentation model based on character word classification.