General Language Research Articles

DKA-RG: Disease-Knowledge-Enhanced Fine-Grained Image–Text Alignment for Automatic Radiology Report Generation

Automatic radiology report generation is a task that combines artificial intelligence and medical information processing, and it fully relies on computer vision and natural language processing techniques. Nowadays, automatic radiology report generation is still a very challenging task because it requires semantically adequate alignment of data from two modalities: radiology images and text. Existing approaches tend to focus on coarse-grained alignment at the global level and do not take into account the disease characteristics of radiology images at fine-grained semantics, which results in the generated reports potentially omitting key disease diagnostic descriptions. In this work, we propose a new approach, disease-knowledge-enhanced fine-grained image–text alignment for automatic radiology report generation (DKA-RG). The method combines global and disease-level alignment, thus facilitating the extraction of fine-grained disease features by the model. Our approach also introduces a knowledge graph to inject medical domain expertise into the model. Our proposed DKA-RG consists of two training steps: the image–report alignment stage and the image-to-report generation stage. In the alignment stage, we use global contrastive learning to align images and texts from a high level and also augment disease contrastive learning with medical knowledge to enhance the disease detection capability. In the report generation stage, the report text generated from the images is more accurate in describing the disease information thanks to sufficient alignment. Through extensive quantitative and qualitative experiments on two widely used datasets, we validate the effectiveness of our DKA-RG on the task of radiology report generation. Our DKA-RG achieves superior performance on multiple types of metrics (natural language generation and clinical efficacy metrics) compared to existing methods, demonstrating that the method can improve the reliability and accuracy of automatic radiology report generation systems.

Enhancing AI Responses in Chemistry: Integrating Text Generation, Image Creation, and Image Interpretation through Different Levels of Prompts

Enhancing AI Responses in Chemistry: Integrating Text Generation, Image Creation, and Image Interpretation through Different Levels of Prompts

Even laypeople use legalese

Whereas principles of communicative efficiency and legal doctrine dictate that laws be comprehensible to the common world, empirical evidence suggests legal documents are largely incomprehensible to lawyers and laypeople alike. Here, a corpus analysis (n = 59) million words) first replicated and extended prior work revealing laws to contain strikingly higher rates of complex syntactic structures relative to six baseline genres of English. Next, two preregistered text generation experiments (n = 286) tested two leading hypotheses regarding how these complex structures enter into legal documents in the first place. In line with the magic spell hypothesis, we found people tasked with writing official laws wrote in a more convoluted manner than when tasked with writing unofficial legal texts of equivalent conceptual complexity. Contrary to the copy-and-edit hypothesis, we did not find evidence that people editing a legal document wrote in a more convoluted manner than when writing the same document from scratch. From a cognitive perspective, these results suggest law to be a rare exception to the general tendency in human language toward communicative efficiency. In particular, these findings indicate law's complexity to be derived from its performativity, whereby low-frequency structures may be inserted to signal law's authoritative, world-state-altering nature, at the cost of increased processing demands on readers. From a law and policy perspective, these results suggest that the tension between the ubiquity and impenetrability of the law is not an inherent one, and that laws can be simplified without a loss or distortion of communicative content.

Large language models (LLMs): survey, technical frameworks, and future challenges

Artificial intelligence (AI) has significantly impacted various fields. Large language models (LLMs) like GPT-4, BARD, PaLM, Megatron-Turing NLG, Jurassic-1 Jumbo etc., have contributed to our understanding and application of AI in these domains, along with natural language processing (NLP) techniques. This work provides a comprehensive overview of LLMs in the context of language modeling, word embeddings, and deep learning. It examines the application of LLMs in diverse fields including text generation, vision-language models, personalized learning, biomedicine, and code generation. The paper offers a detailed introduction and background on LLMs, facilitating a clear understanding of their fundamental ideas and concepts. Key language modeling architectures are also discussed, alongside a survey of recent works employing LLM methods for various downstream tasks across different domains. Additionally, it assesses the limitations of current approaches and highlights the need for new methodologies and potential directions for significant advancements in this field.

Navigating the Ethical Landscape of ChatGPT Integration in Scientific Research: Review of Challenges and Recommendations

This study aims to investigate the transformative potential of incorporating ChatGPT into academic research while addressing consequential challenges and ethical concerns. The chatbot, as a large language model, allows users to interact with it, providing advantages in the academic research context. The benefits of real-time interaction, information synthesis, and language generation have the potential to modernize how researchers engage with and extract insights from vast amounts of data. Additionally, ChatGPT demonstrates a strong command of programming languages, which is highly relevant in modern research related to machine learning and artificial intelligence. However, it still has certain limitations, such as the potential for generating fake citations and issues related to plagiarism. Users need to be aware of these limitations. Therefore, researchers must be careful regarding the potential and limitations of ChatGPT, as it can improve efficiency but may decrease the quality of work if misused. This study recommends guidelines in academic research for using LLMs like ChatGPT. Overall, it’s not ChatGPT that downgrades the quality of academic work; rather, it’s the irresponsible use of ChatGPT that does it.

Comprehensive Review on Natural Language Generation for Automated Report Writing in Finance

The financial industry is transforming with the advent of Natural Language Generation (NLG), a subset of Natural Language Processing (NLP), which automates data conversion into coherent and contextually relevant narratives. This paper presents a comprehensive review of NLG's application in financial report automation, tracing its evolution from template-based methods to advanced deep learning and knowledge graph techniques. We discuss the relevance of NLG in automating report generation, its role in enhancing data analysis and decision-making, and its potential to improve investor communications and compliance with regulations. The paper identifies research gaps, including the need for optimization, accuracy improvement, and the integration of machine learning models for better classification and prediction. A proposed methodology for structured report generation is outlined, leveraging deep learning architectures such as RNNs and LSTMs. Future work aims to address these gaps and further integrate NLG into financial reporting, promising to streamline processes, reduce costs, and provide more personalized and insightful financial narratives.

Comparative analysis of paraphrasing performance of ChatGPT, GPT‐3, and T5 language models using a new ChatGPT generated dataset: ParaGPT

AbstractParaphrase generation is a fundamental natural language processing (NLP) task that refers to the process of generating a well‐formed and coherent output sentence that exhibits both syntactic and/or lexical diversity from the input sentence, while simultaneously ensuring that the semantic similarity between the two sentences is preserved. However, the availability of high‐quality paraphrase datasets has been limited, particularly for machine‐generated sentences. In this paper, we present ParaGPT, a new paraphrase dataset of 81,000 machine‐generated sentence pairs, including 27,000 reference sentences (ChatGPT‐generated sentences), and 81,000 paraphrases obtained by using three different large language models (LLMs): ChatGPT, GPT‐3, and T5. We used ChatGPT to generate 27,000 sentences that cover a diverse array of topics and sentence structures, thus providing diverse inputs for the models. In addition, we evaluated the quality of the generated paraphrases using various automatic evaluation metrics. Furthermore, we provide insights into the strengths and drawbacks of each LLM in generating paraphrases by conducting a comparative analysis of the paraphrasing performance of the three LLMs. According to our findings, ChatGPT's performance, as per the evaluation metrics provided, was deemed impressive and commendable, owing to its higher‐than‐average scores for semantic similarity, which implies a higher degree of similarity between the generated paraphrase and the reference sentence, and its relatively lower scores for syntactic diversity, indicating a greater diversity of syntactic structures in the generated paraphrase. ParaGPT is a valuable resource for researchers working on NLP tasks like paraphrasing, text simplification, and text generation. We make the ParaGPT dataset publicly accessible to researchers, and as far as we are aware, this is the first paraphrase dataset produced based on ChatGPT.

Generative artificial intelligence: a systematic review and applications

AbstractIn recent years, the study of artificial intelligence (AI) has undergone a paradigm shift. This has been propelled by the groundbreaking capabilities of generative models both in supervised and unsupervised learning scenarios. Generative AI has shown state-of-the-art performance in solving perplexing real-world conundrums in fields such as image translation, medical diagnostics, textual imagery fusion, natural language processing, and beyond. This paper documents the systematic review and analysis of recent advancements and techniques in Generative AI with a detailed discussion of their applications including application-specific models. Indeed, the major impact that generative AI has made to date, has been in language generation with the development of large language models, in the field of image translation and several other interdisciplinary applications of generative AI. Moreover, the primary contribution of this paper lies in its coherent synthesis of the latest advancements in these areas, seamlessly weaving together contemporary breakthroughs in the field. Particularly, how it shares an exploration of the future trajectory for generative AI. In conclusion, the paper ends with a discussion of Responsible AI principles, and the necessary ethical considerations for the sustainability and growth of these generative models.

Bias and Fairness in Large Language Models: A Survey

Abstract Rapid advancements of large language models (LLMs) have enabled the processing, understanding, and generation of human-like text, with increasing integration into systems that touch our social sphere. Despite this success, these models can learn, perpetuate, and amplify harmful social biases. In this article, we present a comprehensive survey of bias evaluation and mitigation techniques for LLMs. We first consolidate, formalize, and expand notions of social bias and fairness in natural language processing, defining distinct facets of harm and introducing several desiderata to operationalize fairness for LLMs. We then unify the literature by proposing three intuitive taxonomies, two for bias evaluation, namely, metrics and datasets, and one for mitigation. Our first taxonomy of metrics for bias evaluation disambiguates the relationship between metrics and evaluation datasets, and organizes metrics by the different levels at which they operate in a model: embeddings, probabilities, and generated text. Our second taxonomy of datasets for bias evaluation categorizes datasets by their structure as counterfactual inputs or prompts, and identifies the targeted harms and social groups; we also release a consolidation of publicly available datasets for improved access. Our third taxonomy of techniques for bias mitigation classifies methods by their intervention during pre-processing, in-training, intra-processing, and post-processing, with granular subcategories that elucidate research trends. Finally, we identify open problems and challenges for future work. Synthesizing a wide range of recent research, we aim to provide a clear guide of the existing literature that empowers researchers and practitioners to better understand and prevent the propagation of bias in LLMs.

Clinical Applications of Generative Artificial Intelligence in Radiology: Image Translation, Synthesis and Text Generation

Clinical Applications of Generative Artificial Intelligence in Radiology: Image Translation, Synthesis and Text Generation

Utilizing language models for advanced electrocardiogram analysis

Electrocardiography (ECG) signals are often referred to as the language of the heart and have been widely utilized for diagnosing various heart ailments, particularly arrhythmias. Consequently, numerous machine learning models have been employed to automatically detect heart disorders using ECG signals. In this research, the primary objective is to detect arrhythmias using a center-symmetric self-organized textual pattern. A novel feature engineering model has been introduced, which includes the following components: (i) multilevel feature extraction using the proposed center-symmetric self-organized textual pattern (CSSOTP), (ii) iterative neighborhood component analysis (INCA), and (iii) classification using k-nearest neighbors (kNN) with 10-fold cross-validation. During the feature extraction phase, a multilevel feature extraction model incorporating maximum absolute pooling (MAP) and the proposed CSSOTP was applied. The CSSOTP was designed to select the most appropriate pattern for the given data block. A large language model (LLM) was leveraged to generate text for creating patterns, and ChatGPT was used to assist in text generation. To identify the most informative features, the INCA feature selector was employed, and the selected features were subsequently classified using the kNN classifier. An impressive 96.20 % classification accuracy was achieved by the CSSOTP-based feature engineering model when tested on an ECG dataset containing 17 classes. Furthermore, comparisons with state-of-the-art feature engineering models were conducted, demonstrating that the proposed model has superior classification capabilities.

University policies on generative AI in Asia: promising practices, gaps, and future directions

ABSTRACT Considering the opportunities and challenges raised by generative AI (GenAI) technologies, many universities have been developing policies to guide the integration of GenAI in their academic community. Focusing on Asian universities, this paper presents a scoping review of policy development regarding the GenAI integration. Guided by the theoretical framework of technology integration, this study examined the GenAI policies of 30 universities from the QS top 60 Asian universities with inductive content analysis. The review reveals that these policies prioritized text generation applications, student management, and academic integrity, suggesting an effort to uphold traditional academic values and encourage informed adoption. While the universities were at different stages of understanding and managing GenAI, they tended towards a comprehensive approach in formatting their guidance for internal stakeholders and assessment policies. The review also uncovered gaps in policymaking, such as the exclusion of non-academic staff, limited use of evidence-based practices, international misalignments, and a strong adherence to traditional academic paradigms. This scoping review provides a comprehensive and multifaceted overview of policy development in Asian universities, laying the groundwork for global discussions about the role of GenAI in higher education.

A Novel Rational Medicine Use System Based on Domain Knowledge Graph

Medication errors, which could often be detected in advance, are a significant cause of patient deaths each year, highlighting the critical importance of medication safety. The rapid advancement of data analysis technologies has made intelligent medication assistance applications possible, and these applications rely heavily on medical knowledge graphs. However, current knowledge graph construction techniques are predominantly focused on general domains, leaving a gap in specialized fields, particularly in the medical domain for medication assistance. The specialized nature of medical knowledge and the distinct distribution of vocabulary between general and biomedical texts pose challenges. Applying general natural language processing techniques directly to the medical domain often results in lower accuracy due to the inadequate utilization of contextual semantics and entity information. To address these issues and enhance knowledge graph production, this paper proposes an optimized model for named entity recognition and relationship extraction in the Chinese medical domain. Key innovations include utilizing Medical Bidirectional Encoder Representations from Transformers (MCBERT) for character-level embeddings pre-trained on Chinese biomedical corpora, employing Bi-directional Gated Recurrent Unit (BiGRU) networks for extracting enriched contextual features, integrating a Conditional Random Field (CRF) layer for optimal label sequence output, using the Piecewise Convolutional Neural Network (PCNN) to capture comprehensive semantic information and fusing it with entity features for better classification accuracy, and implementing a microservices architecture for the medication assistance review system. These enhancements significantly improve the accuracy of entity relationship classification in Chinese medical texts. The model achieved good performance in recognizing most entity types, with an accuracy of 88.3%, a recall rate of 85.8%, and an F1 score of 87.0%. In the relationship extraction stage, the accuracy reached 85.7%, the recall rate 82.5%, and the F1 score 84.0%.

Utilizing large language models in infectious disease transmission modelling for public health preparedness

IntroductionOpenAI's ChatGPT, a Large Language Model (LLM), is a powerful tool across domains, designed for text and code generation, fostering collaboration, especially in public health. Investigating the role of this advanced LLM chatbot in assisting public health practitioners in shaping disease transmission models to inform infection control strategies, marks a new era in infectious disease epidemiology research. This study used a case study to illustrate how ChatGPT collaborates with a public health practitioner in co-designing a mathematical transmission model. MethodsUsing natural conversation, the practitioner initiated a dialogue involving an iterative process of code generation, refinement, and debugging with ChatGPT to develop a model to fit 10 days of prevalence data to estimate two key epidemiological parameters: i) basic reproductive number (Ro) and ii) final epidemic size. Verification and validation processes are conducted to ensure the accuracy and functionality of the final model. ResultsChatGPT developed a validated transmission model which replicated the epidemic curve and gave estimates of Ro of 4.19 (95 % CI: 4.13- 4.26) and a final epidemic size of 98.3 % of the population within 60 days. It highlighted the advantages of using maximum likelihood estimation with Poisson distribution over least squares method. ConclusionIntegration of LLM in medical research accelerates model development, reducing technical barriers for health practitioners, democratizing access to advanced modeling and potentially enhancing pandemic preparedness globally, particularly in resource-constrained populations.

DialogueNeRF: towards realistic avatar face-to-face conversation video generation

Conversation is an essential component of virtual avatar activities in the metaverse. With the development of natural language processing, significant breakthroughs have been made in text and voice conversation generation. However, face-to-face conversations account for the vast majority of daily conversations, while most existing methods focused on single-person talking head generation. In this work, we take a step further and consider generating realistic face-to-face conversation videos. Conversation generation is more challenging than single-person talking head generation, because it requires not only the generation of photo-realistic individual talking heads, but also the listener’s response to the speaker. In this paper, we propose a novel unified framework based on the neural radiance field (NeRF) to address these challenges. Specifically, we model both the speaker and the listener with a NeRF framework under different conditions to control individual expressions. The speaker is driven by the audio signal, while the response of the listener depends on both visual and acoustic information. In this way, face-to-face conversation videos are generated between human avatars, with all the interlocutors modeled within the same network. Moreover, to facilitate future research on this task, we also collected a new human conversation dataset containing 34 video clips. Quantitative and qualitative experiments evaluate our method in different aspects, e.g., image quality, pose sequence trend, and natural rendering of the scene in the generated videos. Experimental results demonstrate that the avatars in the resulting videos are able to carry on a realistic conversation, and maintain individual styles.

Knowledge-aware audio-grounded generative slot filling for limited annotated data

Manually annotating fine-grained slot-value labels for task-oriented dialogue (ToD) systems is an expensive and time-consuming endeavour. This motivates research into slot-filling methods that operate with limited amounts of labelled data. Moreover, the majority of current work on ToD is based solely on text as the input modality, neglecting the additional challenges of imperfect automatic speech recognition (ASR) when working with spoken language. In this work, we propose a Knowledge-Aware Audio-Grounded generative slot filling framework, termed KA2G, that focuses on few-shot and zero-shot slot filling for ToD with speech input. KA2G achieves robust and data-efficient slot filling for speech-based ToD by (1) framing it as a text generation task, (2) grounding text generation additionally in the audio modality, and (3) conditioning on available external knowledge (e.g. a predefined list of possible slot values). We show that combining both modalities within the KA2G framework improves the robustness against ASR errors. Further, the knowledge-aware slot-value generator in KA2G, implemented via a pointer generator mechanism, particularly benefits few-shot and zero-shot learning. Experiments, conducted on the standard speech-based single-turn SLURP dataset and a multi-turn dataset extracted from a commercial ToD system, display strong and consistent gains over prior work, especially in few-shot and zero-shot setups.

Crystal Composition Transformer: Self-Learning Neural Language Model for Generative and Tinkering Design of Materials.

Self-supervised neural language models have recently achieved unprecedented success from natural language processing to learning the languages of biological sequences and organic molecules. These models have demonstrated superior performance in the generation, structure classification, and functional predictions for proteins and molecules with learned representations. However, most of the masking-based pre-trained language models are not designed for generative design, and their black-box nature makes it difficult to interpret their design logic. Here a Blank-filling Language Model for Materials (BLMM) Crystal Transformer is proposed, a neural network-based probabilistic generative model for generative and tinkering design of inorganic materials. The model is built on the blank-filling language model for text generation and has demonstrated unique advantages in learning the "materials grammars" together with high-quality generation, interpretability, and data efficiency. It can generate chemically valid materials compositions with as high as 89.7% charge neutrality and 84.8% balanced electronegativity, which are more than four and eight times higher compared to a pseudo-random sampling baseline. The probabilistic generation process of BLMM allows it to recommend materials tinkering operations based on learned materials chemistry, which makes it useful for materials doping. The model is applied to discover a set of new materials as validated using the Density Functional Theory (DFT) calculations. This work thus brings the unsupervised transformer language models based generative artificial intelligence to inorganic materials. A user-friendly web app for tinkering materials design has been developed and can be accessed freely at www.materialsatlas.org/blmtinker.

Questions Analysis of the General English language Exam for the Sixth Vocational Grade in Light of Bloom’s Taxonomy

The present study aims at identify the dominant cognitive six levels emphasized in the general English language exams for sixth vocational grade students. It also explores the potential implications of the exam question types for effective teaching practices in the sixth vocational grade English language context. To achieve the purpose of the study adopted a qualitative approach, utilizing document analysis (exam questions). Tools of the study includes a content analysis form designed in the light of six levels of Bloom taxonomies (remembering- understanding- applying- analysis- evaluating- creating). The study is limited on the exam of general English in 5 vocational departments in three academic years (2020-2021, 2021-2022, 2022-2023). Results revealed the increase of questions that assess lower cognitive levels of Blooms taxonomies in all the academic years of all branches and the sacristy of the questions that assess high cognitive levels; one or two at most in each year. More questions needed for each branch to assess analyzing, critical and interpretive levels of comprehension. Creativity and more writing tasks needed based on the curriculum objectives in each branch.

Components of Mathematical Language and Mathematical Ability in 5-Year-Old Dual Language Learners

ABSTRACT A growing body of research suggests that children’s understanding of mathematical language is critical for their mathematical abilities. Most of this work has been restricted to single language learners (i.e., SLLs), and used dual language learners (i.e., DLLs) as an exclusion criterion, raising questions about the generalizability of these findings to DLLs. This is particularly important given the observed differences in the association between general language and mathematics for DLLs as compared to SLLs. In our study, conducted in Belgium, we therefore investigated the associations between components of mathematical language and different mathematical abilities in 75 DLLs and compared them to 75 age- and sex-matched SLLs (M age = 4 years 8 months; 33 females in each group). Unfortunately, the two groups could not be matched on socioeconomic status (i.e., SES), and SES-levels of the parents of DLLs were significantly lower than SES-levels of SLLs. We assessed children’s quantitative and spatial language, their mathematical abilities (numerical competencies, geometry, and measurement), as well as their general vocabulary and spatial skills. All children, including DLLs, were only tested in the language of instruction (i.e., Dutch) in which DLLs exhibited varying levels of proficiency. Results revealed that DLLs obtained lower scores than SLLs in quantitative and spatial language, mathematical abilities, general vocabulary, but not in spatial skills. Associations between components of mathematical language and mathematical abilities were similar across DLLs and SLLs, except for the relation between spatial language and geometry, which was weaker in DLLs.

Easy-read and large language models: on the ethical dimensions of LLM-based text simplification

The production of easy-read and plain language is a challenging task, requiring well-educated experts to write context-dependent simplifications of texts. Therefore, the domain of easy-read and plain language is currently restricted to the bare minimum of necessary information. Thus, even though there is a tendency to broaden the domain of easy-read and plain language, the inaccessibility of a significant amount of textual information excludes the target audience from partaking or entertainment and restricts their ability to live life autonomously. Large language models can solve a vast variety of natural language tasks, including the simplification of standard language texts to easy-read or plain language. Moreover, with the rise of generative models like GPT, easy-read and plain language may be applicable to all kinds of natural language texts, making formerly inaccessible information accessible to marginalized groups like, a.o., non-native speakers, and people with mental disabilities. In this paper, we argue for the feasibility of text simplification and generation in that context, outline the ethical dimensions, and discuss the implications for researchers in the field of ethics and computer science.