Word Segmentation Research Articles

Semi-Supervised Chinese Word Segmentation in Geological Domain Using Pseudo-Lexicon and Self-Training Strategy

Chinese word segmentation (CWS), which involves splitting the sequence of Chinese characters into words, is a key task in natural language processing (NLP) for Chinese. However, the complexity and flexibility of geologic terms require that domain-specific knowledge be utilized in CWS for geoscience domains. Previous studies have identified several challenges that have an impact on CWS in the geoscience domain, including the absence of abundant labeled data and difficult-to-delineate complex geological word boundaries. To solve these problems, a novel semi-supervised deep learning framework, GeoCWS, is developed for CWS in the geoscience domain. The framework is designed with domain-enhanced features and an uncertainty-aware self-training strategy. First, n-grams are automatically constructed from the input text as a pseudo-lexicon. Then, a backbone model is suggested that learns domain-enhanced features by introducing a pseudo-lexicon-based memory mechanism to delineate complex geological word boundaries based on BERT. Next, the backbone model is fine-tuned with a small amount of labeled data to obtain the teacher model. Finally, we design a self-training strategy with joint confidence and uncertainty awareness to improve the generalization ability of the backbone model to unlabeled data. Our method outperformed the state-of-the-art baseline methods in extensive experiments, and ablation experiments verified the effectiveness of the proposed backbone model and self-training strategy.

The influences of social support expressed from doctors and disclosed from peers on patient decision-making: an analysis from the online health community

Patients are turning to OHCs to deal with the stresses and complications of infertility. As a stigmatized disease, infertile patients may value informational support and emotional support differently, which is ignored in existing studies. Based on social support theory, this study aims to reveal the role of informational support and emotional support of doctors in infertile patient decision-making. We select Haodf.com as our data source and we collected information for all new patients of 2,989 doctors. we use Jieba word segmentation tool and TF-IDF algorithm to segment word and find keywords. Then random forest algorithm is used to classify 750 training texts and 250 test texts. We measure all the treatment experience of these new patients between September and December 2022 and then take the average value (TreatExpi). The sentiment analysis application program interface from Baidu AI-NLP is used to measure emotion state of patients and then the average value is included (EmoStai). In order to eliminate the influence of other factors, control variables including the number of patients online (#Patienti), the recommendation (Recommendationi), hospital level (Hleveli), doctor medical title (Mtitlei) and service price (Serpi). First, the effects of social support expressed from doctor and social support disclosed from peers show heterogeneity on patient decision-making. Second, the influences of informational support and emotional support also show difference. Third, the above relationships change with the characteristics of patients. Detecting different types and different sources of social support activities via text mining contributes to between understanding patient decision behavior in OHCs. This study contributes social support theory and OHC studies, and helps the management and design OHCs.

The role of conscious attention in auditory statistical learning: Evidence from patients with impaired consciousness.

The need for attention to enable statistical learning is debated. Testing individuals with impaired consciousness offers valuable insight, but very few studies have been conducted due to the difficulties inherent in such studies. Here, we examined the ability of patients with varying levels of disorders of consciousness (DOC) to extract statistical regularities from an artificial language composed of randomly concatenated pseudowords by measuring frequency tagging in EEG. The objectives were firstly, to assess the automaticity of the segmentation process and the correlations between the level of covert consciousness and statistical learning capacities; secondly, to identify potential new diagnostic indicators. We observed that segmentation abilities were preserved in some minimally conscious patients, suggesting that auditory statistical learning is an inherently automatic low-level process. Due to significant inter-individual variability, word segmentation might not be robust enough for clinical use. In contrast, temporal accuracy of auditory syllable responses correlates strongly with coma severity.

Word Vector Representation of Latin Cuengh Based on Root Feature Enhancement

The Latin Cuengh is a kind of language used in China’s minority areas. Due to its complex pronunciation and semantic system, it is difficult to spread widely. To deal with and protect this language further, this paper considers using the current word vector representation technology to study it. Word vector representation is the basic method and an important foundation of current research on natural language processing. It relies on a large number of data resources and is obtained through the paradigm of pre-training and feature learning. Due to the extreme lack of Latin Cuengh corpus resources, it is very difficult to obtain word vectors by relying on a large amount of data training. In this study, we propose a word vector representation method that combines the root features of Latin Cuengh words. Specifically, while training and learning the Latin Cuengh language corpus, this method uses the special word roots in the Latin Cuengh language to modify the training process, which can enhance the expression ability of the root features. The method uses the mask method based on BERT to mask the word roots after word segmentation and predict the masked word roots in the output layer of the model to obtain a better vector representation of Latin Cuengh words. The experimental results show that the word vector representation method proposed in this paper is effective and has the ability to express Latin Cuengh semantics. The accuracy rate of words semantic is nearly 2% points higher than that of BERT representation, and the judgment of the semantic similarity of words is more accurate.

A corpus of Chinese word segmentation agreement

The absence of explicit word boundaries is a distinctive characteristic of Chinese script, setting it apart from most alphabetic scripts, leading to word boundary disagreement among readers. Previous studies have examined how this feature may influence reading performance. However, further investigations are required to generate more ecologically valid and generalizable findings. In order to advance our understanding of the impact of word boundaries in Chinese reading, we introduce the Chinese Word Segmentation Agreement (CWSA) corpus. This corpus consists of 500 sentences, comprising 9813 character tokens and 1590 character types, and provides data on word segmentation agreement at each character position. The data revealed a high level of overall segmentation agreement (92%). However, participants disagreed on the position of word boundaries in 8.96% of the cases. Moreover, about 85% of the sentences contained at least one ambiguous word boundary. The character strings with high levels of disagreement were tentatively classified into three categories, namely the morphosyntactic type (e.g., “反映–了”), modifier–head type (e.g., “科學–教育”), and others (e.g., “大力–支持”). Finally, the agreement scores also significantly influenced reading behaviors, as evidenced by analyses with published eye movement data. Specifically, a high level of disagreement was associated with longer single fixation durations. We discuss the implications of these results and highlight how the CWSA corpus can facilitate future research on word segmentation in Chinese reading.

Improving Chinese word segmentation with character–lexicon class attention

Improving Chinese word segmentation with character–lexicon class attention

Low-Resource Chinese Named Entity Recognition via CNN-based Multitask Learning

Named entity recognition (NER) is a fundamental subtask for information extraction that aims to locate and classify named entities in unstructured text into predefined categories. Recently, large-scale language models (LLMs) have achieved SOTA performance on a variety of natural language processing tasks. However, because NER is a sequence labeling task in nature while LLMs is a text-generation model, the performance of LLMs on NER is still significantly below supervised baselines, and NER remains a difficult task. Meanwhile, the word boundary and semantic information of Chinese words are usually quite vague, as words contained in Chinese texts are not separated by spaces. Thus, the NER task still requires supervised learning paradigm and heavily relies on large amounts of labeled data, such as entity type and boundary information. However, the cost of labeling data can be prohibitively large, and the purely supervised approaches usually suffer from poor generalization capability. In this article, we propose a multitask learning-based bidirectional iterated dilated convolution model, BCNN-CWS, for low-resource NER via leveraging word boundary information of Chinese word segmentation (CWS) task. Specifically, to efficiently recognize named entities, an iterated dilated convolutional model with a limited number of layers is implemented. In addition, a bidirectional causal convolution mechanism is presented for contextual information extraction. Results of extensive experiments on public Chinese datasets demonstrate that BCNN-CWS achieves superior performance over state-of-the-art models, and it yields up to about 50% speed improvement over existing methods. It is worth noting that BCNN-CWS can be further improved by combining with a pretrained model. Received: 25 Spetember 2024 | Revised: 4 November 2024 | Accepted: 28 November 2024 Conflicts of Interest The authors declare that they have no conflicts of interest to this work. Data Availability Statement The data that support the findings of this study are openly available in GitLab at https://github.com/jiangfeng13/BCNN-CWS Author Contribution Statement Tao Wu: Conceptualization, Methodology, Writing – original draft, Writing – review & editing, Visualization, Supervision. Xinwen Cao: Resources, Data curation. Feng Jiang: Software, Validation, Formal analysis, Investigation, Writing – original draft. Canyixing Cui: Data curation, Writing -review & editing. Xuehao Li: Resources. Xingping Xian: Supervision, Project administration, Funding acquisition.

UnifiedCut: A Simple and Efficient Neural Model for Thai, Burmese and Khmer Word Segmentation

Word segmentation is a critical task in natural language processing for southeast Asian Abugida languages, including Thai, Burmese, and Khmer. Existing approaches demonstrate that models using fixed-length windowed context inputs can achieve high segmentation accuracy; however, they often rely on low-level character features or language-specific preprocessing. Character-based methods can limit feature learning, while language-specific features add complexity due to specialized preprocessing requirements. This paper introduces UnifiedCut, which is a neural model that leverages multiple n-grams within a windowed multi-head attention mechanism. This design captures segmentation features from local contexts and multi-perspective n-gram inputs, enhancing generalization and recall, particularly for out-of-vocabulary words. Compared to CNN- and RNN-based approaches, UnifiedCut’s multi-head attention enables finer-grained feature extraction and greater parallelism, resulting in a faster, more scalable solution. Comprehensive experiments on public datasets for Thai, Burmese, and Khmer show that UnifiedCutachieves state-of-the-art performance in word segmentation.

MAFD: Multiple Adversarial Features Detector for Enhanced Detection of Text-Based Adversarial Examples

Adversarial attacks in the field of Natural Language Processing greatly undermine the effectiveness and safety of models, raising significant challenges when it comes to real-world implementation. The researchers suggested using detection methods to identify and reject hostile samples while maintaining the accuracy of the original model. Nevertheless, current detection methods depend on analyzing a single characteristic, resulting in restricted resilience and flexibility. To address these constraints, we proposed the Multiple Adversarial Features Detector (MAFD), an innovative detection technique that utilizes a wide range of adversarial features, such as segmented perplexity, word frequency, and probability distribution, to enhance the effectiveness of detecting adversarial examples. Our comprehensive experiments shows that MAFD outperforms existing advanced methods in terms of detection accuracy and displays significant robustness and adaptability when applied to various base detectors and attack scenarios. In addition, the design of MAFD facilitates the seamless integration of further adversarial features, hence enhancing its detection capabilities.

Exploring the role of word segmentation on parafoveal processing during Chinese reading

ABSTRACT The importance of the word as a unit of meaning is well-established for readers of both alphabetic languages and Chinese. However, the unspaced nature of written Chinese raises questions about how readers use upcoming information to guide word segmentation and to adjust the parafoveal processing of subsequent characters. Using an eye-tracking experiment, we investigated whether Chinese readers pre-process character C2 more when it forms a word with C1 than when they belong to separate words. The boundary paradigm was used to manipulate the preview of C2, such that readers saw either an identity (normal) or pseudo-character preview. Linear mixed-effects models revealed reduced preview benefit when C1 and C2 were separate words. These results suggest that despite the absence of visual segmentation cues, Chinese readers are able to utilise the parafoveal preview to support the identification of word boundaries and modulate the extent of their parafoveal processing to prioritise the processing of word units.

PromptCNER: A Segmentation-based Method for Few-shot Chinese NER with Prompt-tuning

Recognizing Chinese entities in low-resource settings is a challenging but promising task, which extracts structured pre-defined entities and corresponding types from unstructured text. Compared with the prosperous Named Entity Recognition (NER) methods for Indo-European languages, such as English, the research on Chinese NER is still in its infancy. The main obstacles to the development of Chinese NER methods include the ambiguity of Chinese entity boundary recognition and limited data resources. To address these issues, in this paper, a word-segmentation-based model is present for few-shot Chinese NER. First, we enumerate all possible candidate entity spans on the character level for accurate entity boundary identification with the proposed word segmentation and combination strategy. Then, one kind of question-answer-based prompt template loaded with the candidate entity spans is proposed to cast entity extraction into the masked token prediction task, for dealing with the low-data problem by taking full advantage of the generality and transferability of the pre-trained language model. The extensive experimental results show that our method outperforms the state-of-the-art baselines in low-data settings and also achieves comparable performance in full-data settings.

The construction of a Chinese fine-grained sentiment dictionary for Chinese domestic investors (CN-FSD) and its application

Stock bar comment texts are used to forecast stock market trends because they contain sentiment data provided by investors. However, previous research has relied on generic or polar sentiment dictionaries to construct sentiment indicators, which has limited progress in this field. This paper crawls nearly 1.8 million stock bar comment texts from the Eastmoney (www.eastmoney.com) Stock Bar Forum to perform sentiment analysis in this specialized area of the financial market. It reclassifies existing Chinese sentiment dictionaries, and utilizes Word2Vec and CNN models and word frequency statistics with a penalty mechanism to extract polar sentiment words. Additionally, the improved PMI (Pointwise Mutual Information) algorithm was utilized for fine-grained segmentation of negative emotion words. Finally, a fine-grained sentiment dictionary (four categories: happy, anger, sad and fear; with a total of 6155 words) was established for Chinese domestic investors (denoted by CN-FSD). We found that the improved PMI algorithm increased accuracy by 14%. In addition, we introduce the investor sentiment quantification indicators from two aspects, i.e., fine-grained sentiments and on the emotion theory dimension of psychology from the valence and arousal dimensions. This is done to verify the validity and robustness of the constructed dictionary. The case study from 2019 to 2022 discovered that under the influence of extreme occurrences like COVID-19, the fine-grained sentiment indicators constructed in this study are found to be significantly more strongly correlated with market trend indicators like index return and trading volume than the existing polar or fine-grained sentiment dictionaries. ‘fear’ emotions in fine-grained emotions dominate the stock market. Just as this case study emphasizes the importance of ‘fear’ sentiment, studying the indicators of high arousal sentiment is also important. This case study further validates the efficacy of CN-FSD.

Distilling Knowledge in Machine Translation of Agglutinative Languages with Backward and Morphological Decoders

Agglutinative languages often have morphologically complex words(MCWs) composed of multiple morphemes arranged in a hierarchical structure, posing significant challenges in translation tasks. We present a novel Knowledge Distillation approach tailored for improving the translation of such languages. Our method involves an encoder, a forward decoder, and two auxiliary decoders: a backward decoder and a morphological decoder. The forward decoder generates target morphemes autoregressively and is augmented by distilling knowledge from the auxiliary decoders. The backward decoder incorporates future context, while the morphological decoder integrates target-side morphological information. We have also designed a reliability estimation method to selectively distill only the reliable knowledge from these auxiliary decoders. Our approach relies on morphological word segmentation. We show that the word segmentation method based on unsupervised morphology learning outperforms the commonly used Byte Pair Encoding method on highly agglutinative languages in translation tasks. Our experiments conducted on English-Tamil, English-Manipuri, and English-Marathi datasets show that our proposed approach achieves significant improvements over strong Transformer-based NMT baselines.

The Influence of Speed on Error Rates in Simultaneous Interpreting: An Observational Study (English to Arabic)

In simultaneous interpreting, the rate of source speech (SR) plays an important role in the quality of the interpreter. As the rate of speech increases, interpreters have to process and translate words faster, which can lead to more errors. This phenomenon is because faster speech harms the interpreter's attention and work, causing more omissions, additions, and other linguistic and paralinguistic errors. The impact of these errors can affect the accuracy and quality of interpretation, which highlights the need to understand how changes in speech rate affect performance and the production of good strategies to solve problems. This study examines how the rate of speech (SR) affects the quality of simultaneous interpreting (SI), particularly to assess the impact of the rate of speech on the interpreter. For this purpose, a method was adopted that was divided into seven categories based on Barik's (1971) classification. These categories include segment omission, word and phrase-level omission, deviation of meaning, addition, incomplete sentence, full pause, and long pause. In this study, the performance of six United Nations interpreters with many years of experience was evaluated using six audios ranging from 124 to 164 words per minute (wpm). Findings indicated that higher speaking frequency led to more errors, especially in word and segment omissions. This emphasizes the importance of addressing specific strategies to overcome the challenges that rapid speech can pose to the accuracy and effectiveness of simultaneous interpreting.

TeaTFactor: a prediction tool for tea plant transcription factors based on BERT.

A transcription factor (TF) is a sequence-specific DNA-binding protein, which plays key roles in cell-fate decision by regulating gene expression. Predicting TFs is key for tea plant research community, as they regulate gene expression, influencing plant growth, development, and stress responses. It is a challenging task through wet lab experimental validation, due to their rarity, as well as the high cost and time requirements. As a result, computational methods are increasingly popular to be chosen. The pre-training strategy has been applied to many tasks in natural language processing (NLP) and has achieved impressive performance. In this paper, we present a novel recognition algorithm named TeaTFactor that utilizes pre-training for the model training of TFs prediction. The model is built upon the BERT architecture, initially pre-trained using protein data from UniProt. Subsequently, the model was fine-tuned using the collected TFs data of tea plants. We evaluated four different word segmentation methods and the existing state-of-the-art prediction tools. According to the comprehensive experimental results and a case study, our model is superior to existing models and achieves the goal of accurate identification. In addition, we have developed a web server at http://teatfactor.tlds.cc, which we believe will facilitate future studies on tea transcription factors and advance the field of crop synthetic biology.

Principle and Application for Rumination Computing Algorithms

To fully analyze, mine, and utilize the information and knowledge implied in problem resolving use cases, this paper proposed the autonomous learning method based on machine inducting, hypothesis formulating, and result verifying, which was similar to the biological process of cows ruminating, called rumination computing. Firstly, after inducting and summarizing over 1080 mathematic application problem, the system architecture and general algorithm for humanoid automatic resolving mathematic application problems were represented, which typically included functional modules such as commonsense knowledge base, domain knowledge base, and local knowledge base, preprocessing, word segmentation and part of speech tagging, semantic framework matching, global semantic analyzing, thinking mechanism implementing, etc. Secondly, after the use case solutions were approved, three typical rumination computing modes, including vocabulary sequence, semantic relationship, and computing action, were introduced based on the correct results, resolving steps, and basic rumination actions. The rumination computing step plan was formulated, new knowledge was obtained from the commonsense and results verification, so the continuous autonomous learning loop for machine thinking was formed. Detailed explanations were provided for the three core algorithms implemented (rumination framework algorithm, rumination semantic algorithm, rumination action algorithm). Then, by specific mathematic application problem humanoid resolving user cases, the above three types of rumination computing modes were illumined.

Arabic consonant length perception depends on the relative speech rate of the distal context.

Listeners adapt to the speech rate of talkers. Many studies of speech rate adaptation have focused on the influence of rate information on the perception of word segmentation or segmental perception in English. The effects of immediately adjacent (i.e., proximal) information are generally strong on the perception of both segments and segmentation, but the effects of rate information temporally remote from (i.e., distal to) ambiguous speech signals are less clear, especially for segments. The present study examines the influence of distal rate adaptation on the perception of geminate consonants in Arabic that straddle a morpheme boundary (i.e., heteromorphemic geminates). Participants heard sentences that at one point were ambiguous to the presence of the Arabic definite clitic al, which, under certain circumstances, can be realized as gemination of the subsequent word-initial consonant. The sentences were either recorded with or without the clitic and with three possible distal speech rates in the context of the clitic. They transcribed the sentences and reported what they heard, and those transcriptions were analyzed for the contribution of the original recording and the distal speech rate on the perception of al. It was found that the perception of geminates in Arabic is rate dependent. This extends the knowledge of the effects of distal rate cues to Arabic, showing that Arabic geminate consonants are perceived relative to the rate of the distal context.

Corporate innovation and performance: a textual measure based on securities analysts’ research reports

ABSTRACT The relationship between innovation investment and firm performance has been a central focus of academic research. Traditional metrics, such as patent counts and R&D expenditures, are often criticised for their limited scope and lack of diversity. In contrast, analyst reports offer a richer source of information on innovation activities. This study employs textual analysis to assess firms’ innovation efforts by examining research reports on publicly listed companies. Using methods such as word segmentation, the research constructs corporate innovation indicators based on the LDA (Latent Dirichlet Allocation) topic model. Analyzing data from 2010 to 2021, the study empirically validates these new innovation indicators and investigates their impact on firm performance. The findings indicate that corporate innovation significantly enhances firm performance. Compared to traditional metrics like patents, the indicators developed here are more comprehensive, offering a broader measure of innovation and addressing the limitations of conventional metrics. These results are consistent across various robustness checks. Additionally, the study finds that corporate innovation improves performance primarily by increasing total factor productivity. Furthermore, strong corporate governance amplifies the positive impact of innovation on firm performance.

BiLSTM-Filt: Neural network for radar word segmentation

Radar word extraction is the analysis foundation for multi-function radars (MFRs) in electronic intelligence (ELINT). Although neural networks enhance performance in radar word extraction, current research still faces challenges from complex electromagnetic environments and unknown radar words. Therefore, in this paper, we propose a promising two-stage radar word extraction framework, consisting of segmentation and recognition. To fill the vacancy of radar word segmentation, we establish the mathematical model from the time series analysis viewpoint and design a novel segmentation neural network based on Bi-direction Long Short-Term Memory with a filter module (BiLSTM-Filt). Specific radar word structure characteristics are extracted by training the network and applied for detecting radar words in the pulse train. To further improve segmentation performance, a bounding box regression method is designed to merge information from sub-region structures. Simulation experiments on a typical MFR, Mercury, reveal that the proposed method can outperform the baseline methods within complex electromagnetic environments, containing corrupted environments, various pulse backgrounds, and variable pulse train lengths. Due to the artificial design structure, the proposed method can also make a trial on unknown radar word segmentation.

Chinese word segmentation with heterogeneous graph convolutional network

Abstract Recently, deep learning methods have achieved remarkable success in the Chinese word segmentation (CWS) task. Some of them enhance the CWS model by utilizing contextual features and external resources (e.g., sub-words, lexicon, and syntax). However, existing approaches fail to fully use the heterogeneous features and their structural information. Therefore, in this paper, we propose a heterogeneous information learning framework for CWS, named heterogeneous graph neural segmenter (HGNSeg), which exploits heterogeneous features with the graph convolutional networks and the pretrained language model. Experimental results on six benchmark datasets (e.g., SIGHAN 2005 and SIGHAN 2008) confirm that HGNSeg can effectively improve the performance of CWS. Importantly, HGNSeg also demonstrates an excellent ability to alleviate the out-of-vocabulary (OOV) issue in cross-domain scenarios.