Natural Language Model Research Articles

Text2EL + : Expert Guided Event Log Enrichment using Unstructured Text

Through the application of process mining, business processes can be improved on the basis of process execution data captured in event logs. Naturally, the quality of this data determines the quality of the improvement recommendations. Improving data quality is non-trivial and there is great potential to exploit unstructured text, e.g. from notes, reviews, and comments, for this purpose and to enrich event logs. To this end, this paper introduces Text2EL + a three-phase approach to enrich event logs using unstructured text. In its first phase, events and (case and event) attributes are derived from unstructured text linked to organisational processes. In its second phase, these events and attributes undergo a semantic and contextual validation before their incorporation in the event log. In its third and final phase, recognising the importance of human domain expertise, expert guidance is used to further improve data quality by removing redundant and irrelevant events. Expert input is used to train a Named Entity Recognition (NER) model with customised tags to detect event log elements. The approach applies natural language processing techniques, sentence embeddings, training pipelines and models, as well as contextual and expression validation. Various unstructured clinical notes associated with a healthcare case study were analysed and completeness, concordance, and correctness of the derived event log elements were evaluated through experiments. The results show that the proposed method is feasible and applicable.

ChatGPT in The Lab: Analyzing the Potential Pros and Cons for Research Professionals

ChatGPT is a natural large language model developed by OpenAI, which generates human-like responses to a wide range of questions and prompts. For researchers, ChatGPT can provide quick and accurate answers to academic questions. The study delves into the nuanced landscape and scrutinizes the benefits and pitfalls associated with the usage of ChatGPT among higher education researchers. Based on a convenient sampling, the study involves a mixed-methods approach that includes both qualitative and quantitative data collection and analysis. One thousand ninety-nine researchers, who have used ChatGPT for various academic projects, are the subjects under study. The data was collected via online questionnaires and face-to-face interviews from four Moroccan universities and higher education institutions: Sultan Moulay Slimane University, Sidi Mohamed Ben Abdellah University, Moulay Ismail University, and Al Akhawayn University. The research data is processed and analyzed through the SPSS software and analyzed to discern patterns and trends in ChatGPT usage. The findings of the study underscore the multifaceted impact of ChatGPT in academic research. The outcomes show that the benefits of ChatGPT include its ability to provide quick and accurate responses to queries, facilitate learning and information-seeking, and enhance communication skills. In contrast, the pitfalls include its potential to lead to an overreliance on technology, reduced critical thinking skills, and the risk of plagiarism. Such a result has significant implications for the integration of ChatGPT into academic research practices. While acknowledging its advantages, researchers are urged to approach its use with caution, emphasizing the importance of maintaining a balance between technological assistance and the preservation of essential cognitive skills. The study provides insights into the ongoing discourse on the role of Chatbots in higher education and underscores the need for researchers to incorporate strategies that harness the benefits of ChatGPT while mitigating its potential drawbacks.

MOBFinder: a tool for mobilization typing of plasmid metagenomic fragments based on a language model.

Mobilization typing (MOB) is a classification scheme for plasmid genomes based on their relaxase gene. The host ranges of plasmids of different MOB categories are diverse, and MOB is crucial for investigating plasmid mobilization, especially the transmission of resistance genes and virulence factors. However, MOB typing of plasmid metagenomic data is challenging due to the highly fragmented characteristics of metagenomic contigs. We developed MOBFinder, an 11-class classifier, for categorizing plasmid fragments into 10 MOB types and a nonmobilizable category. We first performed MOB typing to classify complete plasmid genomes according to relaxase information and then constructed an artificial benchmark dataset of plasmid metagenomic fragments (PMFs) from those complete plasmid genomes whose MOB types are well annotated. Next, based on natural language models, we used word vectors to characterize the PMFs. Several random forest classification models were trained and integrated to predict fragments of different lengths. Evaluating the tool using the benchmark dataset, we found that MOBFinder outperforms previous tools such as MOBscan and MOB-suite, with an overall accuracy approximately 59% higher than that of MOB-suite. Moreover, the balanced accuracy, harmonic mean, and F1-score reached up to 99% for some MOB types. When applied to a cohort of patients with type 2 diabetes (T2D), MOBFinder offered insights suggesting that the MOBF type plasmid, which is widely present in Escherichia and Klebsiella, and the MOBQ type plasmid might accelerate antibiotic resistance transmission in patients with T2D. To the best of our knowledge, MOBFinder is the first tool for MOB typing of PMFs. The tool is freely available at https://github.com/FengTaoSMU/MOBFinder.

PolyNC: a natural and chemical language model for the prediction of unified polymer properties.

Language models exhibit a profound aptitude for addressing multimodal and multidomain challenges, a competency that eludes the majority of off-the-shelf machine learning models. Consequently, language models hold great potential for comprehending the intricate interplay between material compositions and diverse properties, thereby accelerating material design, particularly in the realm of polymers. While past limitations in polymer data hindered the use of data-intensive language models, the growing availability of standardized polymer data and effective data augmentation techniques now opens doors to previously uncharted territories. Here, we present a revolutionary model to enable rapid and precise prediction of Polymer properties via the power of Natural language and Chemical language (PolyNC). To showcase the efficacy of PolyNC, we have meticulously curated a labeled prompt-structure-property corpus encompassing 22 970 polymer data points on a series of essential polymer properties. Through the use of natural language prompts, PolyNC gains a comprehensive understanding of polymer properties, while employing chemical language (SMILES) to describe polymer structures. In a unified text-to-text manner, PolyNC consistently demonstrates exceptional performance on both regression tasks (such as property prediction) and the classification task (polymer classification). Simultaneous and interactive multitask learning enables PolyNC to holistically grasp the structure-property relationships of polymers. Through a combination of experiments and characterizations, the generalization ability of PolyNC has been demonstrated, with attention analysis further indicating that PolyNC effectively learns structural information about polymers from multimodal inputs. This work provides compelling evidence of the potential for deploying end-to-end language models in polymer research, representing a significant advancement in the AI community's dedicated pursuit of advancing polymer science.

Sentiment polarity analysis of love letters: Evaluation of TextBlob, Vader, Flair, and Hugging Face transformer

Sentiment analysis is the task of computationally identifying and quantifying the emotions and opinions expressed in text. However, existing sentiment analysis tools, while increasingly sophisticated, face challenges when applied to complex and personal domains such as love letters. This study investigates the performance and accuracy of four popular Python libraries for sentiment analysis (TextBlob, Vader, Flair, and Hugging Face Transformer) in determining the polarity and intensity of sentiments in love letters. A corpus of 300 love letters was collected and randomly sampled to provide 500 sentences for analysis. Due to the lack of labelled data, human experts participated in evaluating the quality and accuracy of the sentiment annotations. Inter-rater agreements were computed among four judges across randomly sampled sentence lots in two distinct blind rounds. The results reveal varying degrees of effectiveness and agreement among sentiment analysis tools (TextBlob, Vader, Flair, and Hugging Face) and human judges, with Cohen?s Kappa values showing low to moderate agreement (ranging from 0.09 to 0.77), and each tool demonstrating unique strengths?Vader excelling in sentiment intensity and Flair with Hugging Face better at contextual nuances?in handling the emotional complexity of the texts. The study also highlights limitations and proposes some custom metrics for evaluating sentiment analysis tools in the context of love letters, such as tenderness index, passion quotient, nostalgia score, and others. The findings contribute to the emerging field of sentiment analysis and provide insights for developing natural language models better suited for personal and emotionally charged domains.

Reviving Static Charts into Live Charts.

Data charts are prevalent across various fields due to their efficacy in conveying complex data relationships. However, static charts may sometimes struggle to engage readers and efficiently present intricate information, potentially resulting in limited understanding. We introduce "Live Charts," a new format of presentation that decomposes complex information within a chart and explains the information pieces sequentially through rich animations and accompanying audio narration. We propose an automated approach to revive static charts into Live Charts. Our method integrates GNN-based techniques to analyze the chart components and extract data from charts. Then we adopt large natural language models to generate appropriate animated visuals along with a voice-over to produce Live Charts from static ones. We conducted a thorough evaluation of our approach, which involved the model performance, use cases, a crowd-sourced user study, and expert interviews. The results demonstrate Live Charts offer a multi-sensory experience where readers can follow the information and understand the data insights better. We analyze the benefits and drawbacks of Live Charts over static charts as a new information consumption experience.

Data Set Formation Method for Checking the Quality of Learning Language Models of the Transitive Relation in the Logical Conclusion Problem Context

A method for data set formation has been developed to verify the ability of pre-trained models to learn transitivity dependencies. The generated data set was used to test the quality of learning the transitivity dependencies in the task of natural language inference (NLI). Testing of a data set with a size of 10,000 samples (MultiNLI) used to test the RoBerta model. It was found that this model is good at studying transitive dependencies in the task of logical inference because all samples from the formed dataset were correctly classified as belonging to the class similar, contradiction and neutral. It was also investigated that in the task of logical inference, the class similarity is more directed than contradiction and neutral. Because if the premise and hypothesis in the data set are swapped, the accuracy of the RoBerta model decreases by a factor of 2.97,1.17,1.26 for the similar (0.98→0.33) , neutral (0.90→0.77) , and contradiction (0.98→0.78) classes, respectively. The study iteration time is 0.0028 seconds, so only half of the data set requires approximately 84 hours of collection. This research is relevant because the ability of natural language models to explore such dependencies as transitivity, which is not explicitly specified in the training data set, is an important element of the model’s ability to generalize. It was found that RoBerta’s model is good at studying transitive dependencies in the logical inference task because it correctly classified belonging to the class similar, contradiction, and neutral on all samples from the generated data set.

Semantic Technologies to Model Battery Data and Knowledge

Data-intensive battery research is here to stay. With open data incentives on the rise and simpler software interfaces granting access to powerful machine learning algorithms, researchers are uncovering a wealth of insights from battery data. However, data without context are just unusable digital bits. Battery data must be appropriately described to render it interoperable and reusable for traditional and data-intensive research. This requires overcoming numerous challenges associated to data heterogeneity, inconsistency, standardization, and meaning. Semantic technologies address these challenges by providing a unified and flexible framework for integrating and reasoning about data from heterogeneous sources.In this contribution, we will introduce Semantic Technologies and how they enable both human and software agents to understand and reuse battery data. We will present the tools we are developing to i) encourage the adoption of a common standard vocabulary to describe the battery domain[1], ii) connect research resources (datasets, experts, reports, publications, projects) within an accessible platform[2], and iii) reduce the effort required to upgrade data with semantic context. Overall, these tools take concrete steps towards guaranteeing compliance with FAIR data principles. We will further outline plans for future development, including strategies for reaching out to the global battery landscape, facilitate the curation of battery data for data-driven research, and enable knowledge exploration via natural language models.[1] S. Clark et al., "Toward a Unified Description of Battery Data," Adv. Energy Mater., vol. 2102702, 2021.[2] https://github.com/BIG-MAP/Demo-BatteryDataSemanticSearch

Beyond human in neurosurgical exams: ChatGPT's success in the Turkish neurosurgical society proficiency board exams

Chat Generative Pre-Trained Transformer (ChatGPT) is a sophisticated natural language model that employs advanced deep learning techniques and is trained on extensive datasets to produce responses akin to human conversation for user inputs. In this study, ChatGPT's success in the Turkish Neurosurgical Society Proficiency Board Exams (TNSPBE) is compared to the actual candidates who took the exam, along with identifying the types of questions it answered incorrectly, assessing the quality of its responses, and evaluating its performance based on the difficulty level of the questions. Scores of all 260 candidates were recalculated according to the exams they took and included questions in those exams for ranking purposes of this study. The average score of the candidates for a total of 523 questions is 62.02 ± 0.61 compared to ChatGPT, which was 78.77. We have concluded that in addition to ChatGPT's higher response rate, there was also a correlation with the increase in clarity regardless of the difficulty level of the questions with Clarity 1.5, 2.0, 2.5, and 3.0. In the participants, however, there is no such increase in parallel with the increase in clarity.

Research on the Change of Talent Cultivation Mode of Undergraduate Law Program of Open University in the Era of Artificial Intelligence

Abstract In the era of artificial intelligence, this paper uses big data analysis to construct a data model of the knowledge system of law, with the help of clustering analysis, cluster students and resources to form a structured knowledge system of law. Combined with the natural language model, which accurately extracts important information, it provides personalized learning resources for students. Eventually, the teaching content is dynamically adjusted according to students’ characteristics, resulting in the integrated design of a personalized teaching mode. The comparison found that the achievement improvement of students using the traditional teaching mode only ranged from 1.8% to 12.6%, and the achievement improvement interval was from 1.9 to 12.6 points. After applying the AI model-based instructional model, the relative improvement in student performance ranged from 19.2% to 19.8%, with an improvement interval of 14.2 to 15.7 points. The students showed a significant teaching effect by demonstrating a 25-point increase in their communication skills performance compared to the control class.

Lingualyzer: A computational linguistic tool for multilingual and multidimensional text analysis

Most natural language models and tools are restricted to one language, typically English. For researchers in the behavioral sciences investigating languages other than English, and for those researchers who would like to make cross-linguistic comparisons, hardly any computational linguistic tools exist, particularly none for those researchers who lack deep computational linguistic knowledge or programming skills. Yet, for interdisciplinary researchers in a variety of fields, ranging from psycholinguistics, social psychology, cognitive psychology, education, to literary studies, there certainly is a need for such a cross-linguistic tool. In the current paper, we present Lingualyzer (https://lingualyzer.com), an easily accessible tool that analyzes text at three different text levels (sentence, paragraph, document), which includes 351 multidimensional linguistic measures that are available in 41 different languages. This paper gives an overview of Lingualyzer, categorizes its hundreds of measures, demonstrates how it distinguishes itself from other text quantification tools, explains how it can be used, and provides validations. Lingualyzer is freely accessible for scientific purposes using an intuitive and easy-to-use interface.

ClotCatcher: a novel natural language model to accurately adjudicate venous thromboembolism from radiology reports

IntroductionAccurate identification of venous thromboembolism (VTE) is critical to develop replicable epidemiological studies and rigorous predictions models. Traditionally, VTE studies have relied on international classification of diseases (ICD) codes which are inaccurate – leading to misclassification bias. Here, we developed ClotCatcher, a novel deep learning model that uses natural language processing to detect VTE from radiology reports.MethodsRadiology reports to detect VTE were obtained from patients admitted to Emory University Hospital (EUH) and Grady Memorial Hospital (GMH). Data augmentation was performed using the Google PEGASUS paraphraser. This data was then used to fine-tune ClotCatcher, a novel deep learning model. ClotCatcher was validated on both the EUH dataset alone and GMH dataset alone.ResultsThe dataset contained 1358 studies from EUH and 915 studies from GMH (n = 2273). The dataset contained 1506 ultrasound studies with 528 (35.1%) studies positive for VTE, and 767 CT studies with 91 (11.9%) positive for VTE. When validated on the EUH dataset, ClotCatcher performed best (AUC = 0.980) when trained on both EUH and GMH dataset without paraphrasing. When validated on the GMH dataset, ClotCatcher performed best (AUC = 0.995) when trained on both EUH and GMH dataset with paraphrasing.ConclusionClotCatcher, a novel deep learning model with data augmentation rapidly and accurately adjudicated the presence of VTE from radiology reports. Applying ClotCatcher to large databases would allow for rapid and accurate adjudication of incident VTE. This would reduce misclassification bias and form the foundation for future studies to estimate individual risk for patient to develop incident VTE.

Automated extraction of domain knowledge in the dairy industry

Three weeks prior to calving to three weeks after calving, the transition period poses challenges for dairy cattle and farmers. Vast changes in housing, feeding, and reproduction might result in milk drop, metabolic and reproductive diseases. Moreover, most of the metabolic processes are intricately linked as many conditions can coexist. This challenge means that dairy producers and their advisors have difficulty drawing concise conclusions because of all aspects and relationships in transition cow management. Herein, machine-learning techniques and knowledge-graph theory were explored with a view to creating a decision-support system that could provide producers and their advisors with knowledge from domain literature. Specifically, knowledge is modelled as entities and relationships in knowledge graph theory, and natural language models were developed to extract information as knowledge graphs. A dataset comprising 1152 sentences from 20 papers was created and split into 922 sentences for training and 230 sentences for testing. Sequentially, two deep learning models were trained to extract entities and relationships respectively. For training results, a Bi-directional Long-Short-Term Memory model was applied for the entity extraction task and obtained an F1 score of 80 %. As for relationship extraction, a Transformer-based model was deployed but yielded a low F1 of 23 %, thus another pre-trained Transformer model with 89 % accuracy was deployed into the system. After feeding the domain literature into the deep-learning models, a knowledge graph of 1,576 nodes and 3,456 edges was constructed and stored in the graph database Neo4j. Afterward, a semantic parsing method was used to allow users to conduct question answering through the knowledge graph in natural language. In addition, to determine the quality of answers that the knowledge built from the papers, answers were sampled and evaluated based on human judgment. On average, answers scored 7.5 out of 10 and proved informative with respect to the original literature. Although the final interactive results demonstrated a high degree of visualization and scalability, this study primarily sought to demonstrate its feasibility. For tailored commercial applications, further improvements could be implemented in knowledge graph expansion and reasoning.

ChatGPT, a brand-new tool to strengthen timeless competencies

In November 2022, the public release of ChatGPT, an artificial intelligence (AI)-based natural language model, was a groundbreaking point in many sectors of human life and education was not the exception. We describe how ChatGPT was integrated in an undergraduate course for an International Relations program in a private Mexican university. Under an action research methodology, we introduced this novel instrument in a course on Future Studies. Students were evaluated on their ability to explain to ChatGPT several discipline-specific methods and to make the AI implement these methods step by step. After six such activities, the outcomes evidenced that the students not only learned how to use the new AI tool and deepen their understanding of prospective methods, but also strengthened three soft or transversal competencies: communication, critical thinking, and logical and methodical reasoning. These results are promising in the framework of Skills for Industry 4.0 and Education for Sustainable Development; even more, they demonstrate how ChatGPT created an opportunity for the students to strengthen, and for the professor to assess, time-tested competencies. This is a call-to action for faculty and educational institutions to incorporate AI in their instructional design, not only to prepare our graduates for professional environments where they will collaborate with these technologies but also to enhance the quality and relevance of higher education in the digital age. Therefore, this work contributes to the growing body of research on how Artificial Intelligence (AI) can be used in higher education settings to enhance learning experiences and outcomes.

Extracting relations from texts using vector language models and a neural network classifier.

The article investigates the possibility of identifying the presence of SKOS (Simple Knowledge Organization System) relations between concepts represented by terms on the base of their vector representation in general natural language models. Several language models of the Word2Vec and GloVe families are considered, on the basis of which an artificial neural network (ANN) classifier of SKOS relations is formed. To train and test the efficiency of the classifier, datasets formed on the basis of the DBPedia and EuroVoc thesauri are used. The experiments performed have shown the high efficiency of the classifier trained using GloVe family models, while training it with use of Word2Vec models looks impossible in the bounds of considered ANN-based classifier architecture. Based on the results, a conclusion is made about the key role of taking into account the global context of the use of terms in the text for the possibility of identifying SKOS relations.

Enhanced protein function prediction by fusion embedding based on protein language models

Natural language models can accomplish non-natural language tasks such as protein prediction, but the actual prediction effect is low and occupies large computational resources. In this paper, a fusion embedding model is proposed to improve the prediction effect of the model and reduce the computational cost of the model by fusing information of different dimensions. The paper is validated by the downstream task of protein function prediction, which provides a reference for solving practical tasks using fusion embedding methods.

Prevalence rate and risk factor analysis of nonalcoholic fatty liver disease in 115 female patients with schizophrenia

Objective: To investigate the incidence rate and risk factors of nonalcoholic fatty liver disease (NAFLD) in patients with schizophrenia (SCZ). Methods: The incidence rate of NAFLD in 115 females with SCZ over 40 years of age with complete clinical data was analyzed with the consent of the Ethics Committee of Nantong Fourth People's Hospital. A physical examination report of healthy subjects (n = 95, female, age 40 years old or older) was taken as the control group. Natural language processing technology was used to extract relevant data from the patient's electronic medical record system. Body mass index, alanine aminotransferase, triglycerides, low-density lipoprotein, leptin, and adiponectin were used to establish a human NAFLD-related model. Logistic regression analysis was used to evaluate the psychiatric symptoms, and physiological and biochemical indexes for the predictive value of NAFLD in female patients with SCZ. Results: The prevalence of NAFLD was significantly higher in the SCZ group (55.7%, 64/115) than that in the control group (26.3%, 25/95) (χ (2) = 18.335, P < 0.001). The prediction model showed that age, alanine aminotransferase, triglycerides, low-density lipoprotein, leptin, adiponectin, and body mass index were significantly correlated with NAFLD in females with SCZ. In the natural language processing search language model, arousal intensity (movements: uncontrolled running behavior) and emotional apathy were strongly linked to female patients with SCZ with NAFLD. Age, alanine aminotransferase, triglycerides, low-density lipoprotein, leptin, and body mass index were risk factors for SCZ to develop NAFLD, and adiponectin levels and uncontrolled running behavior were protective factors. Conclusion: The incidence rate of NAFLD is high in middle-aged and elderly females with SCZ. Natural language processing can help to automatically identify the risk factors for SCZ combined with NAFLD and has predictive and auxiliary diagnostic value.

Testing the limits of natural language models for predicting human language judgements

Testing the limits of natural language models for predicting human language judgements

A parallel optimization and transfer learning approach for summarization in electrical power systems

Transfer learning approaches in natural language processing have been explored and evolved as a potential solution for solving many problems in recent days. The current research on aspect-based summarization shows unsatisfactory accuracy and low-quality generated summaries. Additionally, the potential advantages of combining language models with parallel processing have not been explored in the existing literature. This paper aims to address the problem of aspect-based extractive text summarization using a transfer learning approach and an optimization method based on map reduce. The proposed approach utilizes transfer learning with language models to extract significant aspects from the text. Subsequently, an optimization process using map reduce is employed. This optimization framework includes an in-node mapper and reducer algorithm to generate summaries for important aspects identified by the language model. This enhances the quality of the summary, leading to improved accuracy, particularly when applied to electrical power system documents. By leveraging the strengths of natural language models and parallel data processing techniques, this model presents an opportunity to achieve better text summary generation. The performance metric used is accuracy, measured with the ROUGE tool, incorporating precision, recall and f-measure. The proposed model demonstrates a 6% improvement in scores compared to state-of-the-art techniques.

A deep learning model for natural language querying in Cyber–Physical Systems

As a result of technological advancements, the number of IoT devices and services is rapidly increasing. Due to the increasing complexity of IoT devices and the various ways they can operate and communicate, finding a specific device can be challenging because of the complex tasks they can perform. To help find devices in a timely and efficient manner, in environments where the user may not know what devices are available or how to access them, we propose a recommender system using deep learning for matching user queries in the form of a natural language sentence with Web of Things (WoT) devices or services. The Transformer, a recent attention-based algorithm that gets superior results for natural language problems, is used for the deep learning model. Our study shows that the Transformer can be a recommendation tool for finding relevant WoT devices in Cyber–Physical Systems (CPSs). With hashing as an encoding technique, the proposed model returns the relevant devices with a high grade of confidence. After experimentation, the proposed model is validated by comparing it with our current search system, and the results are discussed. The work can potentially impact real-world application scenarios when many different devices are involved.