Development Of Natural Language Processing Research Articles

Analysis of 2019 Ohio Disease Intervention Specialist (DIS) Records for Syphilis Cases Using Clustering Algorithms.

Developments in natural language processing (NLP) and unsupervised machine learning methodologies (e.g., clustering) have given researchers new tools to analyze both structured and unstructured health data. We applied these methods to 2019 Ohio disease intervention specialist (DIS) syphilis records, to determine whether these methods can uncover novel patterns of co-occurrence of individual characteristics, risk factors, and clinical characteristics of syphilis that are not yet reported in the literature. The 2019 DIS syphilis records (n=1,996) contain both structured data (categorical and numerical variables) and unstructured notes. In the structured data, we examined case demographics, syphilis risk factors, and clinical characteristics of syphilis. For the unstructured text, we applied TF-IDF (term frequency multiplied by inverse document frequency) weights, a common way to convert text into numerical representations. We performed agglomerative clustering with cosine similarity using the CLUTO software. The cluster analysis yielded six clusters of syphilis cases based on patterns in the structured and unstructured data. The average internal similarities were much higher than the average external similarities, indicating that the clusters were well-formed. The factors underlying three of the clusters related to patterns of missing data. The factors underlying the other three clusters were sexual behaviors and partnerships. Notably, one of the three consisted of individuals who reported oral sex with male or anonymous partners while intoxicated, and one was comprised mainly of males who have sex with females. Our analysis resulted in clusters that were well-formed mathematically, but did not reveal novel epidemiological information about syphilis risk factors or transmission that were not already known.

Natural Language, Legal Hurdles: Navigating the Complexities in Natural Language Processing Development and Application

This article delves into the legal challenges faced in developing and deploying Natural Language Processing (NLP) technologies, focusing particularly on the European Union’s legal framework, especially the DSM Directive, the InfoSoc Directive, and the Artificial Intelligence Act. It addresses the legal status and accessibility of language data and the development of NLP technologies under both contractual and exception-based models. The authors acknowledge the partial truth in the saying, “US innovates, China replicates, and the EU regulates”. Although Europe’s AI sector is a global competitor and its strict regulations ensure ethical standards and data protection, these regulations might not necessarily boost competitiveness. Such stringent regulations can introduce complexities that may inhibit innovation relative to regions with more lenient policies.

I Am Not Your Typical Chatbot: Hedonic and Utilitarian Evaluation of Open-Domain Chatbots

With the development of natural language processing (NLP), open-domain chatbots can operate as companions. The success of open-domain chatbots depends on understanding how positive and negative expectancy violations affect both utilitarian and hedonic gratification, which in turn influences user satisfaction and continued use. Therefore, this study examines how violations of expectations influence the hedonic and utilitarian evaluations of open-domain chatbots using the expectancy-violation theory. Participants (n = 204) interacted with an improvising AI chatbot and reported their satisfaction. Results indicated that both hedonic and utilitarian satisfaction were higher when negative expectations were met compared to when expectations were violated. However, no significant difference was found in hedonic gratification between the chatbot’s expected performance and positively unexpected performance. Conversations meeting expectations elicited more utilitarian satisfaction than positively surprising interactions. These findings highlight the dynamics between value types and expectation violations in AI chatbot evaluations.

Identifying COVID-19 Fake News on Social Networks Using Deep Learning: You will not know what happens next!

Fake News has long been an influential source of information on social media, misleading people and causing unpredictable consequences, especially in the COVID-19 era when the spread of Fake News has been amplified. There is a need to develop a platform that can detect Fake News on social platforms. With the development of Natural Language Processing (NLP) and Deep Learning, the detection of misleading information is becoming a reality. In this paper, we propose a feasible optimization scheme that combines and optimizes NLP and Deep Learning to improve the accuracy of Fake News detection. Our model achieves a hit rate of up to 95.3% compared to state-of-the-art techniques. In the proposed system, a GUI-based interface is also designed and developed to facilitate news detection.

A comprehensive review of the application of NLP technology in language learning

The rapid development of Natural Language Processing (NLP) technology has provided new perspectives and tools for the acquisition of second languages. As our world becomes increasingly interconnected, the role of NLP in facilitating language learning has become more prominent. This paper reviews the multifaceted applications of NLP technology in language learning, including auxiliary teaching functions such as reading assistance, writing feedback, oral interaction, and personalized learning. These applications have significantly enhanced language learners' abilities in vocabulary acquisition, grammatical application, pronunciation accuracy, and reading comprehension through real-time feedback, enhanced interactivity, and promotion of cultural understanding. Despite the immense potential of NLP technology in second language learning, challenges such as technical accuracy, cultural adaptability, and data privacy exist. This paper proposes strategies, such as ensuring technical accuracy, curating diverse datasets and safeguarding data privacy to address these challenges. Looking forward, the future development of NLP technology in second language education holds great promise. As NLP continues to evolve, it is expected to contribute to more personalized, effective, and accessible language learning solutions. This paper aims to provide valuable references and insights for educators, technology developers, and policymakers, enabling them to harness the transformative potential of NLP in language education and to navigate the future of educational technology with confidence and foresight.

Building custom NLP tools to annotate discourse-functional features for second language writing research: A tutorial

The current tutorial paper describes a process of developing a custom natural language processing model with a particular focus on a discourse annotation task. After an overview of recent developments in natural language processing (NLP), the paper discusses the development of the Engagement Analyzer (Eguchi & Kyle, 2023), focusing on corpus annotation, the machine learning model, model training, evaluation, and dissemination. A step-by-step tutorial of this process via the spaCy Python package is provided. The paper highlights the feasibility of developing custom NLP tools to enhance the scalability and replicability of the annotation of context-sensitive linguistic features in L2 writing research.

Indonesian-English Textual Similarity Detection Using Universal Sentence Encoder (USE) and Facebook AI Similarity Search (FAISS)

The tremendous development in Natural Language Processing (NLP) has enabled the detection of bilingual and multilingual textual similarity. One of the main challenges of the Textual Similarity Detection (TSD) system lies in learning effective text representation. The research focuses on identifying similar texts between Indonesian and English across a broad range of semantic similarity spectrums. The primary challenge is generating English and Indonesian dense vector representation, a.k.a. embeddings that share a single vector space. Through trial and error, the research proposes using the Universal Sentence Encoder (USE) model to construct bilingual embeddings and FAISS to index the bilingual dataset. The comparison between query vectors and index vectors is done using two approaches: the heuristic comparison with Euclidian distance and a clustering algorithm, Approximate Nearest Neighbors (ANN). The system is tested with four different semantic granularities, two text granularities, and evaluation metrics with a cutoff value of k={2,10}. Four semantic granularities used are highly similar or near duplicate, Semantic Entailment (SE), Topically Related (TR), and Out of Topic (OOT), while the text granularities take on the sentence and paragraph levels. The experimental results demonstrate that the proposed system successfully ranks similar texts in different languages within the top ten. It has been proven by the highest F1@2 score of 0.96 for the near duplicate category on the sentence level. Unlike the near-duplicate category, the highest F1 scores of 0.77 and 0.89 are shown by the SE and TR categories, respectively. The experiment results also show a high correlation between text and semantic granularity.

EnzChemRED, a rich enzyme chemistry relation extraction dataset

Expert curation is essential to capture knowledge of enzyme functions from the scientific literature in FAIR open knowledgebases but cannot keep pace with the rate of new discoveries and new publications. In this work we present EnzChemRED, for Enzyme Chemistry Relation Extraction Dataset, a new training and benchmarking dataset to support the development of Natural Language Processing (NLP) methods such as (large) language models that can assist enzyme curation. EnzChemRED consists of 1,210 expert curated PubMed abstracts where enzymes and the chemical reactions they catalyze are annotated using identifiers from the protein knowledgebase UniProtKB and the chemical ontology ChEBI. We show that fine-tuning language models with EnzChemRED significantly boosts their ability to identify proteins and chemicals in text (86.30% F1 score) and to extract the chemical conversions (86.66% F1 score) and the enzymes that catalyze those conversions (83.79% F1 score). We apply our methods to abstracts at PubMed scale to create a draft map of enzyme functions in literature to guide curation efforts in UniProtKB and the reaction knowledgebase Rhea.

The lived experience of gambling-related harm in natural language.

Gambling-related harms can have a significant negative impact on disordered gamblers, lower risk gamblers, and affected others. Yet, most disordered and lower risk gamblers will never seek formal treatment, often due to the stigma and shame surrounding gambling. Online self-help forums are a popular alternative way for gamblers to anonymously seek help from others. Analysis of these interactions can provide a deeper understanding of gambling than more commonly used research methodologies. In the present study, we leverage recent developments in natural language processing to analyze posts on a U.K.-based online self-help gambling forum. Using correlated topic modeling, we canvass the various types of discussions among forum members. We also combine this approach with semantic similarity analysis based on sentence embeddings, to map first the posts, and then the 10 topics, onto six previously established gambling-related harm domains. The topic modeling revealed a cluster of discussions of many negative emotions, a topic regarding the positive emotions underlying the potential for change, a distinct topic regarding gambling's relationship harms, and numerous environmental factors that contributed to harm. Emotional/psychological and health harms were most strongly associated with users' posts, illustrating the multidimensionality of severe gambling-related harm. Our results reveal the co-occurrence of different harms, such as the frequent mentions of financial harms and concomitant emotional/psychological harms. The analysis of the lived experiences of gambling-related harm in natural language represents a useful tool for gambling research and can provide a different perspective to inform policy. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

DialogueNeRF: towards realistic avatar face-to-face conversation video generation

Conversation is an essential component of virtual avatar activities in the metaverse. With the development of natural language processing, significant breakthroughs have been made in text and voice conversation generation. However, face-to-face conversations account for the vast majority of daily conversations, while most existing methods focused on single-person talking head generation. In this work, we take a step further and consider generating realistic face-to-face conversation videos. Conversation generation is more challenging than single-person talking head generation, because it requires not only the generation of photo-realistic individual talking heads, but also the listener’s response to the speaker. In this paper, we propose a novel unified framework based on the neural radiance field (NeRF) to address these challenges. Specifically, we model both the speaker and the listener with a NeRF framework under different conditions to control individual expressions. The speaker is driven by the audio signal, while the response of the listener depends on both visual and acoustic information. In this way, face-to-face conversation videos are generated between human avatars, with all the interlocutors modeled within the same network. Moreover, to facilitate future research on this task, we also collected a new human conversation dataset containing 34 video clips. Quantitative and qualitative experiments evaluate our method in different aspects, e.g., image quality, pose sequence trend, and natural rendering of the scene in the generated videos. Experimental results demonstrate that the avatars in the resulting videos are able to carry on a realistic conversation, and maintain individual styles.

Optimization techniques for sentiment analysis based on LLM (GPT-3)

With the rapid development of natural language processing (NLP) technology, large-scale pre-trained language models such as GPT-3 have become a popular research object in NLP field. This paper aims to explore sentiment analysis optimization techniques based on large pre-trained language models such as GPT-3 to improve model performance and effect and further promote the development of natural language processing (NLP). By introducing the importance of sentiment analysis and the limitations of traditional methods, GPT-3 and Fine-tuning techniques are introduced in this paper, and their applications in sentiment analysis are explained in detail. The experimental results show that the Fine-tuning technique can optimize GPT-3 model and obtain good performance in sentiment analysis task. This study provides an important reference for future sentiment analysis using large-scale language models.

Challenges of Natural Language Processing from a Linguistic Perspective

As the foundation of artificial intelligence, natural language processing is a research field with great development prospects, closely related to linguistics. Linguistic research has realized the initial natural language processing, and the development of natural language processing has raised higher requirements for linguistic research. Although natural language processing technology has achieved some results, there is still a distance from the most natural human-machine interaction. Language is the expression form of human thinking, and the fundamental purpose of artificial intelligence is to simulate human thinking with computers. To achieve this goal, it is necessary to start with simulating human natural language. This paper mainly analyzes the current challenges of natural language processing from a linguistic perspective, aiming to further improve the level of natural language processing.

Personal Data in Artificial Intelligence Systems: Natural Language Processing Technology

Objective: to conceptualize, from the viewpoint of personal data protection legislation, the development of natural language processing technology, identifying possible legal barriers to such development and directions for further research of the issue.Methods: the research is based on general scientific methods of cognition, along with which formal-legal and comparative-legal methods were applied, as well as the method of theoretical modeling.Results: it was found that the observance of personal data regime natural language processing in the development of natural language processing technology leads technology, to a conflict between private-legal and public-legal interests, which, personal data in turn, creates obstacles for further development of this technology. The shortcomings of the existing legal order are shown, namely, the insufficient correspondence to the technical features of technology development. This may lead to the risks of excessive regulation, or, on the contrary, to the risks of neglecting critical areas that require protection. Problems in qualifying the data involved in the technology development are outlined. An attempt is made to define the limits of ensuring the lawfulness of personal data processing within the natural language processing technology. The material, temporal and territorial effect of the legal regulation in this field is identified as the limits of ensuring the legality. The author touches upon the possibility of using personal data as a consideration, which is important for the development of natural language processing technology and for the improvement of the information and communication technology industry.Scientific novelty: the paper supplements the scientific discussion on the legal regulation of personal data processing by artificial intelligence systems with an analysis of natural language processing technology. The latter is insufficiently studied, making it relevant to research information law, namely, the legal relations arising around artificial intelligence systems, and to assess the impact of a personal data regime on the development of natural language processing technology.Practical relevance: the applied aspects of the problems researched and the results obtained can be used to improve the legal regulation of public relations in the field of creation and development of artificial intelligence, as well as to identify and assess the legal risks arising in the personal data processing by developers of digital products based on natural language processing technology.

Knowledge-enhanced model with dual-graph interaction for confusing legal charge prediction

The rapid development of natural language processing (NLP) technologies has enabled the emergence of legal intelligence assistance systems, with legal charge prediction (LCP) being a critical technology. The automatic LCP aims to determine the final charges based on fact descriptions of criminal cases. LCP assists human judges in managing workloads and improving efficiency, provides accessible legal guidance for individuals, and supports enterprises in litigation financing and compliance monitoring. However, distinguishing between confusing charges in real-world judicial practice remains a significant challenge. Most exist works cannot effectively capture complex relationships and discern subtle differences in fact descriptions while ignoring the legal schematic knowledge. In order to improve confusing LCP performance, we propose a novel knowledge-aware model for legal charge prediction that leverages Graph Neural Networks (GNNs) to capture complex relationships within criminal case descriptions. Specifically, the model constructs structural and semantic graphs from fact descriptions and integrates information from both through a dual-graph interaction process. A legal knowledge transformer generates key knowledge representations at schema and charge levels, while a knowledge matching network incorporates hierarchical charge knowledge into facts. Besides, we also propose two real-world datasets namely Criminal-All and Criminal-Confusing, containing 203 different charges and 86 confusing charges, respectively. To the best of our knowledge, these datasets are the first well-organized datasets for confusing LCP task. Experimental results demonstrate that the proposed model outperforms baselines and significantly improves the distinction of confusing charges, providing valuable support for intelligent legal judgment systems.

Assessing the utility of natural language processing for detecting postoperative complications from free medical text.

Postoperative complication rates are often assessed through administrative data, although this method has proven to be imprecise. Recently, new developments in natural language processing have shown promise in detecting specific phenotypes from free medical text. Using the clinical challenge of extracting four specific and frequently undercoded postoperative complications (pneumonia, urinary tract infection, sepsis, and septic shock), it was hypothesized that natural language processing would capture postoperative complications on a par with human-level curation from electronic health record free medical text. Electronic health record data were extracted for surgical cases (across 11 surgical sub-specialties) from 18 hospitals in the Capital and Zealand regions of Denmark that were performed between May 2016 and November 2021. The data set was split into training/validation/test sets (30.0%/48.0%/22.0%). Model performance was compared with administrative data and manual extraction of the test data set. Data were obtained for 17 486 surgical cases. Natural language processing achieved a receiver operating characteristic area under the curve of 0.989 for urinary tract infection, 0.993 for pneumonia, 0.992 for sepsis, and 0.998 for septic shock, whereas administrative data achieved a receiver operating characteristic area under the curve of 0.595 for urinary tract infection, 0.624 for pneumonia, 0.571 for sepsis, and 0.625 for septic shock. The natural language processing approach was able to capture complications with acceptable performance, which was superior to administrative data. In addition, the model performance approached that of manual curation and thereby offers a potential pathway for complete real-time coverage of postoperative complications across surgical procedures based on natural language processing assessment of electronic health record free medical text.

Rule-Based Text Classification of Dental Diagnosis.

Unstructured medical records boast an abundance of information that could greatly facilitate medical decision-making and improve patient care. With the development of Natural Language Processing (NLP) methodology, the free-text medical data starts to attract more and more research attention. Most existing studies try to leverage the power of such unstructured data using Machine Learning algorithms, which would usually require a relatively large training set, and high computational capacity. However, when faced with a smaller-scale project, opting for an alternative approach may be more effective and practical. This project proposes an efficient and light-weight rule-based approach to categorize dental diagnosis data. It not only fills the void of dental records in the medical free-text processing area, but also demonstrates that with expertly designed research structure and proper implementation, simple method could achieve our study goal very competently.

MSCAN: multi-scale self- and cross-attention network for RNA methylation site prediction

BackgroundEpi-transcriptome regulation through post-transcriptional RNA modifications is essential for all RNA types. Precise recognition of RNA modifications is critical for understanding their functions and regulatory mechanisms. However, wet experimental methods are often costly and time-consuming, limiting their wide range of applications. Therefore, recent research has focused on developing computational methods, particularly deep learning (DL). Bidirectional long short-term memory (BiLSTM), convolutional neural network (CNN), and the transformer have demonstrated achievements in modification site prediction. However, BiLSTM cannot achieve parallel computation, leading to a long training time, CNN cannot learn the dependencies of the long distance of the sequence, and the Transformer lacks information interaction with sequences at different scales. This insight underscores the necessity for continued research and development in natural language processing (NLP) and DL to devise an enhanced prediction framework that can effectively address the challenges presented.ResultsThis study presents a multi-scale self- and cross-attention network (MSCAN) to identify the RNA methylation site using an NLP and DL way. Experiment results on twelve RNA modification sites (m6A, m1A, m5C, m5U, m6Am, m7G, Ψ, I, Am, Cm, Gm, and Um) reveal that the area under the receiver operating characteristic of MSCAN obtains respectively 98.34%, 85.41%, 97.29%, 96.74%, 99.04%, 79.94%, 76.22%, 65.69%, 92.92%, 92.03%, 95.77%, 89.66%, which is better than the state-of-the-art prediction model. This indicates that the model has strong generalization capabilities. Furthermore, MSCAN reveals a strong association among different types of RNA modifications from an experimental perspective. A user-friendly web server for predicting twelve widely occurring human RNA modification sites (m6A, m1A, m5C, m5U, m6Am, m7G, Ψ, I, Am, Cm, Gm, and Um) is available at http://47.242.23.141/MSCAN/index.php.ConclusionsA predictor framework has been developed through binary classification to predict RNA methylation sites.

Aircraft Behavior Recognition on Trajectory Data with a Multimodal Approach

Moving traces are essential data for target detection and associated behavior recognition. Previous studies have used time–location sequences, route maps, or tracking videos to establish mathematical recognition models for behavior recognition. The multimodal approach has seldom been considered because of the limited modality of sensing data. With the rapid development of natural language processing and computer vision, the multimodal model has become a possible choice to process multisource data. In this study, we have proposed a mathematical model for aircraft behavior recognition with joint data manners. The feature abstraction, cross-modal fusion, and classification layers are included in the proposed model for obtaining multiscale features and analyzing multimanner information. Attention has been placed on providing self- and cross-relation assessments on the spatiotemporal and geographic data related to a moving object. We have adopted both a feedforward network and a softmax function to form the classifier. Moreover, we have enabled a modality-increasing phase, combining longitude and latitude sequences with related geographic maps to avoid monotonous data. We have collected an aircraft trajectory dataset of longitude and latitude sequences for experimental validation. We have demonstrated the excellent behavior recognition performance of the proposed model joint with the modality-increasing phase. As a result, our proposed methodology reached the highest accuracy of 95.8% among all the adopted methods, demonstrating the effectiveness and feasibility of trajectory-based behavior recognition.

Developing a Framework to Infer Opioid Use Disorder Severity From Clinical Notes to Inform Natural Language Processing Methods: Characterization Study.

Information regarding opioid use disorder (OUD) status and severity is important for patient care. Clinical notes provide valuable information for detecting and characterizing problematic opioid use, necessitating development of natural language processing (NLP) tools, which in turn requires reliably labeled OUD-relevant text and understanding of documentation patterns. To inform automated NLP methods, we aimed to develop and evaluate an annotation schema for characterizing OUD and its severity, and to document patterns of OUD-relevant information within clinical notes of heterogeneous patient cohorts. We developed an annotation schema to characterize OUD severity based on criteria from the Diagnostic and Statistical Manual of Mental Disorders, 5th edition. In total, 2 annotators reviewed clinical notes from key encounters of 100 adult patients with varied evidence of OUD, including patients with and those without chronic pain, with and without medication treatment for OUD, and a control group. We completed annotations at the sentence level. We calculated severity scores based on annotation of note text with 18 classes aligned with criteria for OUD severity and determined positive predictive values for OUD severity. The annotation schema contained 27 classes. We annotated 1436 sentences from 82 patients; notes of 18 patients (11 of whom were controls) contained no relevant information. Interannotator agreement was above 70% for 11 of 15 batches of reviewed notes. Severity scores for control group patients were all 0. Among noncontrol patients, the mean severity score was 5.1 (SD 3.2), indicating moderate OUD, and the positive predictive value for detecting moderate or severe OUD was 0.71. Progress notes and notes from emergency department and outpatient settings contained the most and greatest diversity of information. Substance misuse and psychiatric classes were most prevalent and highly correlated across note types with high co-occurrence across patients. Implementation of the annotation schema demonstrated strong potential for inferring OUD severity based on key information in a small set of clinical notes and highlighting where such information is documented. These advancements will facilitate NLP tool development to improve OUD prevention, diagnosis, and treatment.

Graph global attention network with memory: A deep learning approach for fake news detection

With the proliferation of social media, the detection of fake news has become a critical issue that poses a significant threat to society. The dissemination of fake information can lead to social harm and damage the credibility of information. To address this issue, deep learning has emerged as a promising approach, especially with the development of Natural Language Processing (NLP). This study introduces a novel approach called Graph Global Attention Network with Memory (GANM) for detecting fake news. This approach leverages NLP techniques to encode nodes with news context and user content. It employs three graph convolutional networks to extract informative features from the news propagation network and aggregates endogenous and exogenous user information. This methodology aims to address the challenge of identifying fake news within the context of social media. Innovatively, the GANM combines two strategies. First, a novel global attention mechanism with memory is employed in the GANM to learn the structural homogeneity of news propagation networks, which is the attention mechanism of a single graph with a history of all graphs. Second, we design a module for partial key information learning aggregation to emphasize the acquisition of partial key information in the graph and merge node-level embeddings with graph-level embeddings into fine-grained joint information. Our proposed method provides a new direction in news detection research with a combination of global and partial information and achieves promising performance on real-world datasets.