Large-scale Text Data Research Articles

Advancing the complex adaptive systems approach to enterprise risk management with quantified risk networks (QRNs)

Business enterprises are complex adaptive systems (CAS) subject to fragility caused by the non-linear effects of uncertain risk events. The regional bank collapses in the United States are a case in point. Recent papers have highlighted a shortcoming of the traditional risk management process, which focuses on compliance but considers neither the interdependence between complex risks nor the mechanism of their non-linear impact on the organization. A new perspective on Enterprise Risk Management (ERM) has instead called for a shift of mindset from mitigating risk to building resilience by honing a CAS view to contextualize, assess, and manage complex risks. But the building blocks needed for a CAS representation of ERM have not yet been systematically developed. Specifically, we are missing a typological inventory of risks and a mapping of risks onto the general structure of an enterprise. In this paper, we build an industry-agnostic inventory of plausible risk factors using information extraction on large scale text data. Additionally, we develop an understanding of risk-function and function-function interdependencies through a survey of top business managers. The result is a novel complex network view of enterprise risk called a Quantified Risk Network (QRN) that displays small-world properties and highlights internal company functions central to non-linear risk propagation mechanisms within the enterprise. The QRN draws attention to vulnerabilities in the enterprise structure such as risk-function connections measured by Edge Betweenness centrality. The generic QRN developed herein is a proof of concept and we advocate that enterprises build their own company-specific QRNs to identify highly connected and central functions in their company structure that could lead to cascading failure when specific risks arise. QRNs can contribute to the objective of building enterprise resilience.

Advanced Computational Methods for News Classification: A Study in Neural Networks and CNN integrated with GPT

In an era inundated with vast amounts of information, the imperative for efficient news classification is paramount. This research explores the sophisticated integration of neural networks and convolutional neural networks (CNN) with Generative Pre-trained Transformers (GPT) to enhance the precision and efficacy of news categorization. The rapid digital dissemination of news necessitates advanced computational methodologies capable of accurate classification and event prediction that include finance and economic events. Leveraging recent advancements in machine learning and natural language processing (NLP), this study utilizes large language models (LLMs) such as GPT and BERT, known for their exceptional comprehension and generation of human-like text. Over 232 days, our methodology classified 33,979 news articles into Education & Learning, Health & Medicine, and Science & Technology, with further subcategorization into 32 distinct subcategories. For evaluation, a sample of 5,000 articles was assessed using metrics such as True Positive (TP), True Negative (TN), False Positive (FP), False Negative (FN), Precision, Recall, and F1-Score. In comparison with the existing studies, the proposed method achieving significantly higher with average scores of 0.986 (Precision), 0.987 (Recall), and 0.987 (F1-Score). This research offers substantial practical contributions, providing detailed insights into news source contributions, effective anomaly detection, and predictive trend analysis using neural networks. The theoretical contributions are profound, demonstrating the mathematical integration of GPT with CNNs and recurrent neural networks. This integration advances computational news classification and exemplifies how sophisticated mathematical frameworks enhance large-scale text data analysis, marking a pivotal advancement in applying advanced computational methods in real-world scenarios.

Natural language processing analysis of the theories of people with multiple sclerosis about causes of their disease

BackgroundWhile potential risk factors for multiple sclerosis (MS) have been extensively researched, it remains unclear how persons with MS theorize about their MS. Such theories may affect mental health and treatment adherence. Using natural language processing techniques, we investigated large-scale text data about theories that persons with MS have about the causes of their disease. We examined the topics into which their theories could be grouped and the prevalence of each theory topic.MethodsA total of 486 participants of the Swiss MS Registry longitudinal citizen science project provided text data on their theories about the etiology of MS. We used the transformer-based BERTopic Python library for topic modeling to identify underlying topics. We then conducted an in-depth characterization of the topics and assessed their prevalence.ResultsThe topic modeling analysis identifies 19 distinct topics that participants theorize as causal for their MS. The topics most frequently cited are Mental Distress (31.5%), Stress (Exhaustion, Work) (29.8%), Heredity/Familial Aggregation (27.4%), and Diet, Obesity (16.0%). The 19 theory topics can be grouped into four high-level categories: physical health (mentioned by 56.2% of all participants), mental health (mentioned by 53.7%), risk factors established in the scientific literature (genetics, Epstein-Barr virus, smoking, vitamin D deficiency/low sunlight exposure; mentioned by 47.7%), and fate/coincidence (mentioned by 3.1%). Our study highlights the importance of mental health issues for theories participants have about the causes of their MS.ConclusionsOur findings emphasize the importance of communication between healthcare professionals and persons with MS about the pathogenesis of MS, the scientific evidence base and mental health.

Public attention and psychological trends towards waste reduction: A large-scale data analysis based on social media

In the digital era, using social media big data to capture the true views and psychological trends shown by the public regarding waste reduction could considerably improve the formulation of targeted waste reduction policies and guide residents to participate in environmental governance from the source side. This study used big data mining technology to trawl 617,771 pieces of waste reduction text from a typical social media site (Sina Microblog). A machine learning algorithm model was used to identify the psychological and cognitive focus of the public and their preferences based on large-scale text data. The temporal and spatial differences in public attention trends, hot topic trends, and sentiment trends were also investigated. The results showed that the public attention level was related to the release of policies by government and that public attention in the southeast coastal areas was higher than that in the northwest inland areas. Moreover, waste reduction had a “working attribute” because the public paid more attention to waste reduction during working hours (i.e., 9:00–12:00 and 15:00–18:00) than during leisure hours. In terms of individual heterogeneity, males were initiators of the topic, whereas females were followers. In particular, participation by young people in waste reduction discussions was higher than for other groups. The topic analysis showed that public attention followed a cooperative construction pattern that had multiple entities, including the individual, community, and government. Overall, the public sentiment score towards waste reduction increased year by year during the study period, with positive sentiment posts accounting for over 70% of the total number of blog posts, and that the vast majority of residents had a positive attitude towards waste reduction. This study expanded current research knowledge by exploring the public attitude response to waste reduction from a social media perspective. The study will help the government to effectively intervene in public behavior tendencies toward waste reduction from the psychological perspective and provided important implications about how the government can enhance its use of social media to effectively guide public opinion and improve policies.

Quantifying advances from basic research to applied research in material science

Exploring the connection between basic science and its practical applications is critical to consider the social justification of the substantial governmental investments in science. However, there's a limited understanding of global research patterns and the dynamics of researcher collaboration from a basic-applied perspective because such studies have been mainly focused only on the biomedical field. Here, the main goal is to propose an indicator to quantify the degree of basic-applied research in academic papers. Using the indicator, we uncover how material science has advanced from basic to applied research, based on the international trends of the indicators and the affiliations of the scientists involved. We develop a methodology that indexes levels of advancement from basic research to applied research based on large-scale text data. The continuous scores assigned to each paper are derived from a vector space embedding technical terms from a broad network data. These scores align with experts' views in material science. This methodology enables us to monitor international trends that China has significantly advanced into applied research, as well as Chinese applied scientists increasingly associating with their domestic institutions. As science and technology implication, our methodology extends the boundary of assessing scientific research on its proximity to real-world applications and provides a tangible measure for funding agencies managing to fund or design research environments.

A systematic evaluation of text mining methods for short texts: Mapping individuals’ internal states from online posts

Short texts generated by individuals in online environments can provide social and behavioral scientists with rich insights into these individuals’ internal states. Trained manual coders can reliably interpret expressions of such internal states in text. However, manual coding imposes restrictions on the number of texts that can be analyzed, limiting our ability to extract insights from large-scale textual data. We evaluate the performance of several automatic text analysis methods in approximating trained human coders’ evaluations across four coding tasks encompassing expressions of motives, norms, emotions, and stances. Our findings suggest that commonly used dictionaries, although performing well in identifying infrequent categories, generate false positives too frequently compared to other methods. We show that large language models trained on manually coded data yield the highest performance across all case studies. However, there are also instances where simpler methods show almost equal performance. Additionally, we evaluate the effectiveness of cutting-edge generative language models like GPT-4 in coding texts for internal states with the help of short instructions (so-called zero-shot classification). While promising, these models fall short of the performance of models trained on manually analyzed data. We discuss the strengths and weaknesses of various models and explore the trade-offs between model complexity and performance in different applications. Our work informs social and behavioral scientists of the challenges associated with text mining of large textual datasets, while providing best-practice recommendations.

Interpretability in Sentiment Analysis: A Self-Supervised Approach to Sentiment Cue Extraction

In this paper, we present a novel self-supervised framework for Sentiment Cue Extraction (SCE) aimed at enhancing the interpretability of text sentiment analysis models. Our approach leverages self-supervised learning to identify and highlight key textual elements that significantly influence sentiment classification decisions. Central to our framework is the development of an innovative Mask Sequence Interpretation Score (MSIS), a bespoke metric designed to assess the relevance and coherence of identified sentiment cues within binary text classification tasks. By employing Monte Carlo Sampling techniques optimized for computational efficiency, our framework demonstrates exceptional effectiveness in processing large-scale text data across diverse datasets, including English and Chinese, thus proving its versatility and scalability. The effectiveness of our approach is validated through extensive experiments on several benchmark datasets, including SST-2, IMDb, Yelp, and ChnSentiCorp. The results indicate a substantial improvement in the interpretability of the sentiment analysis models without compromising their predictive accuracy. Furthermore, our method stands out for its global interpretability, offering an efficient solution for analyzing new data compared to traditional techniques focused on local explanations.

Political Discourse on Twitter: The Case of Election Fraud in South Korea

The fake news that circulated on social media in 2020 and 2022 affected the voters of the South Korean general and presidential elections, respectively. The political controversy, centered on the election fraud that occurred in both the elections, received significant attention from the society. This study emphasizes the Twitter discourse and compares the formation and distribution of election fraud. We conduct semantic social network analysis and structural topic modeling (STM) to represent topics and relationships among the emerging themes using Twitter texts related to both elections. Results demonstrate that discourses on the same political issue exhibited unique contents and structures, as information is formed and distributed differently depending on the type of election in South Korea. Furthermore, this study illustrates the process and analysis of large-scale text data collected from Twitter. It also includes new methods, such as political ideology, for considering an analytical dimension.

Takween: A Comprehensive Arabic Collection and Annotation Tool

In the rapidly evolving field of Natural Language Processing (NLP), the availability of labeled data is crucial for developing and improving machine learning models. Labeled data provides the essential context and structure required for training algorithms to accurately interpret and process human language. However, obtaining labeled data, especially for languages with complex structures like Arabic, presents significant challenges. This paper introduces ”Takween,” a novel tool designed to meet the need for high-quality labeled data in Arabic NLP. Takween leverages social media and Wikipedia as data sources to facilitate the collection of large-scale Arabic text data. The tool enables users to annotate and label the collected data, with functionalities for reviewing annotations and visualizing the labeled data to ensure quality. Additionally, Takween incorporates active learning techniques, which allow for automatically selecting data samples for annotation based on model uncertainty. By integrating data collection, annotation, review, and visualization features with active learning capabilities, Takween offers a comprehensive solution for acquiring high-quality labeled data for Arabic NLP tasks.

Text classification algorithm of tourist attractions subcategories with modified TF-IDF and Word2Vec.

Text classification, as an important research area of text mining, can quickly and effectively extract valuable information to address the challenges of organizing and managing large-scale text data in the era of big data. Currently, the related research on text classification tends to focus on the application in fields such as information filtering, information retrieval, public opinion monitoring, and library and information, with few studies applying text classification methods to the field of tourist attractions. In light of this, a corpus of tourist attraction description texts is constructed using web crawler technology in this paper. We propose a novel text representation method that combines Word2Vec word embeddings with TF-IDF-CRF-POS weighting, optimizing traditional TF-IDF by incorporating total relative term frequency, category discriminability, and part-of-speech information. Subsequently, the proposed algorithm respectively combines seven commonly used classifiers (DT, SVM, LR, NB, MLP, RF, and KNN), known for their good performance, to achieve multi-class text classification for six subcategories of national A-level tourist attractions. The effectiveness and superiority of this algorithm are validated by comparing the overall performance, specific category performance, and model stability against several commonly used text representation methods. The results demonstrate that the newly proposed algorithm achieves higher accuracy and F1-measure on this type of professional dataset, and even outperforms the high-performance BERT classification model currently favored by the industry. Acc, marco-F1, and mirco-F1 values are respectively 2.29%, 5.55%, and 2.90% higher. Moreover, the algorithm can identify rare categories in the imbalanced dataset and exhibit better stability across datasets of different sizes. Overall, the algorithm presented in this paper exhibits superior classification performance and robustness. In addition, the conclusions obtained by the predicted value and the true value are consistent, indicating that this algorithm is practical. The professional domain text dataset used in this paper poses higher challenges due to its complexity (uneven text length, relatively imbalanced categories), and a high degree of similarity between categories. However, this proposed algorithm can efficiently implement the classification of multiple subcategories of this type of text set, which is a beneficial exploration of the application research of complex Chinese text datasets in specific fields, and provides a useful reference for the vector expression and classification of text datasets with similar content.

Valuable Knowledge Mining: Deep Analysis of Heart Disease and Psychological Causes Based on Large-Scale Medical Data

The task of accurately identifying medical entities and extracting entity relationships from large-scale medical text data has become a hot topic in recent years, aiming to mine potential rules and knowledge. How to conduct in-depth context analysis from biomedical texts, such as medical procedures, diseases, therapeutic drugs, and disease characteristics, and identify valuable knowledge in the medical field is our main research content. Through the process of knowledge mining, a deeper understanding of the complex relationships between various factors in diseases can be gained, which holds significant guiding implications for clinical research. An approach based on context semantic analysis is proposed to realize medical entity recognition and entity relationship extraction. In addition, we build a medical knowledge base related to coronary heart disease and combine the NCBI disease dataset and the medical lexicon dataset extracted from the text as the test data of the experiment. Experimental results show that this model can effectively identify entities in medical text data; the WBC model achieved an F1 score of 89.2% in the experiment, while the CSR model achieved an F1 score of 83.4%, and the result is better than other methods.

Blending citizen science with natural language processing and machine learning: Understanding the experience of living with multiple sclerosis.

The emergence of new digital technologies has enabled a new way of doing research, including active collaboration with the public ('citizen science'). Innovation in machine learning (ML) and natural language processing (NLP) has made automatic analysis of large-scale text data accessible to study individual perspectives in a convenient and efficient fashion. Here we blend citizen science with innovation in NLP and ML to examine (1) which categories of life events persons with multiple sclerosis (MS) perceived as central for their MS; and (2) associated emotions. We subsequently relate our results to standardized individual-level measures. Participants (n = 1039) took part in the 'My Life with MS' study of the Swiss MS Registry which involved telling their story through self-selected life events using text descriptions and a semi-structured questionnaire. We performed topic modeling ('latent Dirichlet allocation') to identify high-level topics underlying the text descriptions. Using a pre-trained language model, we performed a fine-grained emotion analysis of the text descriptions. A topic modeling analysis of totally 4293 descriptions revealed eight underlying topics. Five topics are common in clinical research: 'diagnosis', 'medication/treatment', 'relapse/child', 'rehabilitation/wheelchair', and 'injection/symptoms'. However, three topics, 'work', 'birth/health', and 'partnership/MS' represent domains that are of great relevance for participants but are generally understudied in MS research. While emotions were predominantly negative (sadness, anxiety), emotions linked to the topics 'birth/health' and 'partnership/MS' was also positive (joy). Designed in close collaboration with persons with MS, the 'My Life with MS' project explores the experience of living with the chronic disease of MS using NLP and ML. Our study thus contributes to the body of research demonstrating the potential of integrating citizen science with ML-driven NLP methods to explore the experience of living with a chronic condition.

Mobilizing Conceptual Spaces: How Word Embedding Models Can Inform Measurement and Theory Within Organization Science

Word embedding models are a powerful approach for representing the multidimensional conceptual spaces within which communicated concepts relate, combine, and compete with one another. This class of models represent a recent advance in machine learning allowing scholars to efficiently encode complex systems of meaning with minimal semantic distortion based on local and global word co-occurrences from large-scale text data. Although their use has the potential to broaden theoretical possibilities within organization science, embeddings are largely unknown to organizational scholars, where known they have only been mobilized for a narrow set of uses, and they remain unlinked to a theoretical scaffolding that can enable cumulative theory building within the organizations community. Our goal is to demonstrate the promise embedding models hold for organization science by providing a practical roadmap for users to mobilize the methodology in their research and a theoretical guide for consumers of that research to evaluate and conceptually link embedded representations with theoretical significance and potential. We begin by explicitly defining the notions of concept and conceptual space before proceeding to show how these can be represented and measured with word embedding models, noting strengths and weaknesses of the approach. We then provide a set of embedding measurements along with their theoretical interpretation and flexible extension. Our aim is to extract the operational and conceptual significance from technical treatments of word embeddings and place them within a practical, theoretical framework to accelerate research committed to understanding how individuals, teams, and broader collectives represent, communicate, and deploy meaning in organizational life. Supplemental Material: The online appendix is available at https://doi.org/10.1287/orsc.2023.1686 .

Examining Homophily, Language Coordination, and Analytical Thinking in Web-Based Conversations About Vaccines on Reddit: Study Using Deep Neural Network Language Models and Computer-Assisted Conversational Analyses

Vaccine hesitancy has been deemed one of the top 10 threats to global health. Antivaccine information on social media is a major barrier to addressing vaccine hesitancy. Understanding how vaccine proponents and opponents interact with each other on social media may help address vaccine hesitancy. We aimed to examine conversations between vaccine proponents and opponents on Reddit to understand whether homophily in web-based conversations impedes opinion exchange, whether people are able to accommodate their languages to each other in web-based conversations, and whether engaging with opposing viewpoints stimulates higher levels of analytical thinking. We analyzed large-scale conversational text data about human vaccines on Reddit from 2016 to 2018. Using deep neural network language models and computer-assisted conversational analyses, we obtained each Redditor's stance on vaccines, each post's stance on vaccines, each Redditor's language coordination score, and each post or comment's analytical thinking score. We then performed chi-square tests, 2-tailed t tests, and multilevel modeling to test 3 questions of interest. The results show that both provaccine and antivaccine Redditors are more likely to selectively respond to Redditors who indicate similar views on vaccines (P<.001). When Redditors interact with others who hold opposing views on vaccines, both provaccine and antivaccine Redditors accommodate their language to out-group members (provaccine Redditors: P=.044; antivaccine Redditors: P=.047) and show no difference in analytical thinking compared with interacting with congruent views (P=.63), suggesting that Redditors do not engage in motivated reasoning. Antivaccine Redditors, on average, showed higher analytical thinking in their posts and comments than provaccine Redditors (P<.001). This study shows that although vaccine proponents and opponents selectively communicate with their in-group members on Reddit, they accommodate their language and do not engage in motivated reasoning when communicating with out-group members. These findings may have implications for the design of provaccine campaigns on social media.

Exploring the impact of sentiment on multi-dimensional information dissemination using COVID-19 data in China

The outbreak of information epidemic in crisis events, with the channel effect of social media, has brought severe challenges to global public health. Combining information, users and environment, understanding how emotional information spreads on social media plays a vital role in public opinion governance and affective comfort, preventing mass incidents and stabilizing the network order. Therefore, from the perspective of the information ecology and elaboration likelihood model (ELM), this study conducted a comparative analysis based on two large-scale datasets related to COVID-19 to explore the influence mechanism of sentiment on the forwarding volume, spreading depth and network influence of information dissemination. Based on machine learning and social network methods, topics, sentiments, and network variables are extracted from large-scale text data, and the dissemination characteristics and evolution rules of online public opinions in crisis events are further analyzed. The results show that negative sentiment positively affects the volume, depth, and influence compared with positive sentiment. In addition, information characteristics such as richness, authority, and topic influence moderate the relationship between sentiment and information dissemination. Therefore, the research can build a more comprehensive connection between the emotional reaction of network users and information dissemination and analyze the internal characteristics and evolution trend of online public opinion. Then it can help sentiment management and information release strategy when emergencies occur.

OPAL: Ontology-Aware Pretrained Language Model for End-to-End Task-Oriented Dialogue

AbstractThis paper presents an ontology-aware pretrained language model (OPAL) for end-to-end task-oriented dialogue (TOD). Unlike chit-chat dialogue models, task-oriented dialogue models fulfill at least two task-specific modules: Dialogue state tracker (DST) and response generator (RG). The dialogue state consists of the domain-slot-value triples, which are regarded as the user’s constraints to search the domain-related databases. The large-scale task-oriented dialogue data with the annotated structured dialogue state usually are inaccessible. It prevents the development of the pretrained language model for the task-oriented dialogue. We propose a simple yet effective pretraining method to alleviate this problem, which consists of two pretraining phases. The first phase is to pretrain on large-scale contextual text data, where the structured information of the text is extracted by the information extracting tool. To bridge the gap between the pretraining method and downstream tasks, we design two pretraining tasks: ontology-like triple recovery and next-text generation, which simulates the DST and RG, respectively. The second phase is to fine-tune the pretrained model on the TOD data. The experimental results show that our proposed method achieves an exciting boost and obtains competitive performance even without any TOD data on CamRest676 and MultiWOZ benchmarks.

Named entity annotation schema for geological literature mining in the domain of porphyry copper deposits

Owing to the development of natural language processing and deep learning models, geological text data have become a vital resource for knowledge discovery and have attracted the attention of publishers, academic organizations, and domain scientists. However, the extraction of information from unstructured literature still remains a challenge, in which a fundamental issue is the categories and the type of discipline-specific information. This paper presents an effective workflow of building and applying ontologies in geoscience text mining, which includes a use case-driven method for building an ontology model of porphyry copper deposits, an entity annotation schema for text mining, and implementation of them to tackle real-world data. First, the Dexing porphyry copper deposit was selected as a case study to guide the construction of the ontology model. Text data in this study provided a series of entity instances. By analyzing both domain knowledge of mineral deposit models and the instance data, we built classes in the ontology. Second, with the established ontology, a named entity annotation schema comprising 21 entity tokens was designed to scale up the text mining tasks. Third, based on the annotation schema, a draft corpus with more than 200,000 words and a finely corrected corpus of 53,339 words were built for training a geological entity recognizer for porphyry copper deposits. The performance of the geological entity recognizer and the statistical distribution of entities in the corpus prove that the workflow presented in this study is effective for designing entity annotation schemas and facilitating large-scale text data mining in geoscience.

The Real-World Experiences of Persons With Multiple Sclerosis During the First COVID-19 Lockdown: Application of Natural Language Processing.

The increasing availability of "real-world" data in the form of written text holds promise for deepening our understanding of societal and health-related challenges. Textual data constitute a rich source of information, allowing the capture of lived experiences through a broad range of different sources of information (eg, content and emotional tone). Interviews are the "gold standard" for gaining qualitative insights into individual experiences and perspectives. However, conducting interviews on a large scale is not always feasible, and standardized quantitative assessment suitable for large-scale application may miss important information. Surveys that include open-text assessments can combine the advantages of both methods and are well suited for the application of natural language processing (NLP) methods. While innovations in NLP have made large-scale text analysis more accessible, the analysis of real-world textual data is still complex and requires several consecutive steps. We developed and subsequently examined the utility and scientific value of an NLP pipeline for extracting real-world experiences from textual data to provide guidance for applied researchers. We applied the NLP pipeline to large-scale textual data collected by the Swiss Multiple Sclerosis (MS) registry. Such textual data constitute an ideal use case for the study of real-world text data. Specifically, we examined 639 text reports on the experienced impact of the first COVID-19 lockdown from the perspectives of persons with MS. The pipeline has been implemented in Python and complemented by analyses of the "Linguistic Inquiry and Word Count" software. It consists of the following 5 interconnected analysis steps: (1) text preprocessing; (2) sentiment analysis; (3) descriptive text analysis; (4) unsupervised learning-topic modeling; and (5) results interpretation and validation. A topic modeling analysis identified the following 4 distinct groups based on the topics participants were mainly concerned with: "contacts/communication;" "social environment;" "work;" and "errands/daily routines." Notably, the sentiment analysis revealed that the "contacts/communication" group was characterized by a pronounced negative emotional tone underlying the text reports. This observed heterogeneity in emotional tonality underlying the reported experiences of the first COVID-19-related lockdown is likely to reflect differences in emotional burden, individual circumstances, and ways of coping with the pandemic, which is in line with previous research on this matter. This study illustrates the timely and efficient applicability of an NLP pipeline and thereby serves as a precedent for applied researchers. Our study thereby contributes to both the dissemination of NLP techniques in applied health sciences and the identification of previously unknown experiences and burdens of persons with MS during the pandemic, which may be relevant for future treatment.

Improving sequence labeling with labeled clue sentences

Pre-trained language models (PLMs) have achieved noticeable success on a variety of natural language processing tasks, such as sequence labeling. In particular, the existing sequence labeling methods fine-tune PLMs on large-scale labeled data, which can avoid training the sequence labeling models from scratch. The fine-tuning process still requires large amounts of labeled training data so as to be effective. However, obtaining rich annotated data for sequence labeling is a time-consuming and expensive process, creating a substantial barrier for directly applying the PLMs trained on general-purpose large-scale text data to sequence labeling. In this paper, we investigate sequence labeling tasks from a novel perspective and propose a general framework that uses labeled clue sentences to mitigate the problem of insufficient annotation data for sequence labeling. Specifically, we first retrieve the labeled clue sentences for each original sentence in the training set based on the semantic (or syntactic) relevance. Here, the number of annotated clue sentences determines the expansion degree of the training set. Then, we modify the transformer’s self-attention mechanism to not only exploit the contextual information of the original sentence but also leverage the contextual and label information of the labeled clue sentences. In addition, we devise a mask label strategy to further avoid over-fitting by randomly masking out the labels of certain tokens in the clue sentence and then predicting these mask labels based on the context of the tokens corresponding to the mask labels. We verify the effectiveness and generalizability of the proposed framework on three sequence labeling tasks, including Chinese Named Entity Recognition, English Named Entity Recognition, and Aspect Term Extraction. Extensive experimental results show that our method can yield state-of-the-art or competitive results on the three tasks.

Government responsive selectivity and public limited mediation role in air pollution governance: Evidence from large scale text data content mining

Social media has been widely discussed as an informal way for the public to present their environmental appeals. However, evidence on whether and how the government's decision-making will take the public in the non-Western electoral system under consideration remains limited. In this study, we extracted 9.25 million haze-related posts from Sina Weibo and analyzed the topics using the latent Dirichlet allocation topic model. Furthermore, we provide evidence of the Chinese governments’ responsiveness and the role of online public participation in environmental governance. The results show that the Chinese government responds to the public, both online and offline. Specifically, when online public appeal increases by 1%, the local government's posts on social media increase by 0.347%, and investment and regulations increase by 0.0676% and 0.074%, respectively, in the next phase. However, there are selective characteristics of government response: the local government is more concerned about its “political achievements” than undertaking “governance responsibility.” Furthermore, online public participation plays a minor role in the implementation of regulations.