Large Language Models Research Articles

Needs Companion: A Novel Approach to Continuous User Needs Sensing Using Virtual Agents and Large Language Models

In today’s world, services are essential in daily life, and identifying each person’s unique needs is key to creating a human-centered society. Traditional research has used machine learning to recommend services based on user behavior logs without directly detecting individual needs. This study introduces a system called Needs Companion, which automatically detects individual service needs, laying the groundwork for accurate needs sensing. The system defines a needs data model based on the 6W1H framework, uses virtual agents for needs elicitation, and applies large language models (LLMs) to analyze and automatically extract needs. Experiments showed that the system could detect needs accurately and quickly. This research provides interpretable data for personalized services and contributes to fields like machine learning, human-centered design, and requirements engineering.

Towards Responsible Use: Student Perspectives on ChatGPT in Higher Education

The advent of generative artificial intelligence (AI) tools, particularly Large Language Models (LLMs) like ChatGPT, presents a dichotomy of opportunities and challenges within the educational sector. These tools offer the promise of personalized learning support, accessible at any time, thereby enhancing educational engagement and resource availability. However, they also introduce significant concerns, notably the risk of plagiarism, complicating their integration into academic settings. Despite the widespread adoption of ChatGPT by students, there remains a gap in understanding their experiences with these tools and the need for frameworks to guide responsible use. To address this knowledge gap, this study investigates the utilisation of ChatGPT for academic tasks and students' perceptions of its benefits and drawbacks in a university context. The findings reveal that half of the students used ChatGPT either regularly or occasionally for academic tasks. However, only slightly more than one-third of the participants expressed confidence in maintaining the authenticity and originality of their outputs when using ChatGPT. In terms of utility, respondents' perceptions varied, with 25.0% considering it moderately useful, 39.3% very useful, and 10.7% extremely useful for their assignments. Notably, 42.8% observed a significant enhancement in the quality of their work with ChatGPT, compared to assignments completed independently. Despite these advantages, 71.4% of students encountered errors or inaccuracies in ChatGPT's outputs, underscoring the tool's limitations. A significant majority (89.7%) advocate for the establishment of specific guidelines or training to foster effective and responsible usage of ChatGPT in academic contexts. The Chi-square test indicated a significant difference in ChatGPT usage between White and BAME students (p=0.031). These findings underscore the broad utilisation of ChatGPT in student learning and general satisfaction with its application. Nonetheless, students acknowledge the tool's potential for errors and inaccuracies. In light of these challenges, there is a pronounced demand for clear guidelines and proper training to navigate the responsible use of ChatGPT in educational settings. This study illuminates the imperative for educators and institutions to develop strategies that not only leverage the benefits of generative AI tools but also mitigate their risks and promote meaningful learning outcomes.

Medical large language models are susceptible to targeted misinformation attacks

Large language models (LLMs) have broad medical knowledge and can reason about medical information across many domains, holding promising potential for diverse medical applications in the near future. In this study, we demonstrate a concerning vulnerability of LLMs in medicine. Through targeted manipulation of just 1.1% of the weights of the LLM, we can deliberately inject incorrect biomedical facts. The erroneous information is then propagated in the model’s output while maintaining performance on other biomedical tasks. We validate our findings in a set of 1025 incorrect biomedical facts. This peculiar susceptibility raises serious security and trustworthiness concerns for the application of LLMs in healthcare settings. It accentuates the need for robust protective measures, thorough verification mechanisms, and stringent management of access to these models, ensuring their reliable and safe use in medical practice.

Generative AI in Higher Education: Educators' Perspectives on Academic Learning and Integrity

Generative Artificial Intelligence (Gen AI), exemplified by models such as ChatGPT, has rapidly advanced, becoming a significant force in various sectors, including higher education. ChatGPT, a leading application of Gen AI, utilizes large language models (LLMs) to generate human-like text, providing capabilities that range from answering complex questions to facilitating software development. As these tools become increasingly integrated into academic environments, their impact on teaching and learning processes has become more under focus. However, there are not many studies focusing on the impact of such systems on students’ performance and the use of such systems. This study therefore examines the impact of GPT tools on higher education, aiming to address the following research questions: How does the use of GPTs influence the teaching and learning process in higher education? What are the perceived impacts of GPTs on educational practices from university educators’ perspective? The research employs a qualitative methodology, incorporating semi-structured interviews with educators from Swedish universities who have integrated GPT into their curricula. Data were transcribed and analysed using OpenAI's Whisper to extract key themes and insights. Data analysis was done through thematic analysis and categorizing the data using codes. The study uncovers a dual impact of GPT on education, while it offers substantial opportunities for enhancing productivity and personalized learning, it also raises significant concerns about academic integrity, over-reliance on AI, and the potential influences on students’ soft skills. These findings contribute to the discourse on digital learning by highlighting the need for instructional and constructive integration of AI technologies such as GenAI, in educational settings. In addition, the rise and integration of GPT technology is irreversible, and we must adapt to it rather than return to old ways. Embracing AI's potential while addressing its challenges is essential for progress and innovation in this new era. The study emphasizes the importance of developing clear policies and guidelines to ensure that the benefits of GPT are realized without compromising the integrity of the educational process. As such, this research provides valuable insights for educators, policymakers, and scholars interested in the ethical and effective implementation of AI in higher education

Leveraging LLMs for Unstructured Direct Elicitation of Decision Rules

Unstructured Direct Elicitation (UDE) offers a flexible method to capture consumer preferences and decision rules in an unstructured format such as writing an email. However, it relies on subjective human coding and indicative consideration set sizes to make accurate predictions on consideration decisions. This research leverages large language models (LLMs) to replace human judges and make predictions without the need for additional information like indicative consideration set sizes. Empirical analyses show that fine-tuned LLMs effectively interpret decision rules and handle sophisticated considerations in a complex product scenario (automotive study), outperforming the best UDE models by capturing over 25% more information, while their performance in a moderate-scale study on mobile phones is comparable to the best UDE models. The use of LLMs enhances scalability, cost efficiency, and consistency in comprehending unstructured text data and making predictions, offering a promising alternative to human judges and enabling large-scale, real-time implementation of UDE in marketing research and practice. Together with their ability to interact with users, LLMs fine-tuned with representative datasets may serve as a valuable knowledgebase to summarize consumer preferences and decision rules and supply insights for the creation and simulation of marketing strategies.

Utilisation of ChatGPT and other Artificial Intelligence tools among medical faculty in Uganda: a cross-sectional study

Background ChatGPT is an open-source large language model that uses deep learning techniques to generate human-like texts. ChatGPT has the potential to revolutionize medical education as it acts as an interactive virtual tutor and personalized learning assistant. We assessed the use of ChatGPT and other Artificial Intelligence (AI) tools among medical faculty in Uganda. Methods We conducted a descriptive cross-sectional study among medical faculty at four public universities in Uganda from November to December 2023. Participants were recruited consecutively. We used a semi-structured questionnaire to collect data on participants’ socio-demographics and the use of AI tools such as ChatGPT. Our outcome variable was the use of ChatGPT and other AI tools. Data were analyzed in Stata version 17.0. Results We recruited 224 medical faculty, majority [75% (167/224)] were male. The median age (interquartile range) was 41 years (34–50). Almost all medical faculty [90% (202/224)] had ever heard of AI tools such as ChatGPT. Over 63% (120/224) of faculty had ever used AI tools. The most commonly used AI tools were ChatGPT (56.3%) and Quill Bot (7.1%). Fifty-six faculty use AI tools for research writing, 37 for summarizing information, 28 for proofreading work, and 28 for setting exams or assignments. Forty faculty use AI tools for non-academic purposes like recreation and learning new skills. Faculty older than 50 years were 40% less likely to use AI tools compared to those aged 24 to 35 years (Adjusted Prevalence Ratio (aPR):0.60; 95% Confidence Interval (CI): [0.45, 0.80]). Conclusion The use of ChatGPT and other AI tools was high among medical faculty in Uganda. Older faculty (>50 years) were less likely to use AI tools compared to younger faculty. Training on AI use in education, formal policies, and guidelines are needed to adequately prepare medical faculty for the integration of AI in medical education.

Modality deep-learning frameworks for fake news detection on social networks: a systematic literature review

Fake news on social networks is a challenging problem due to the rapid dissemination and volume of information, as well as the ease of creating and sharing content anonymously. Fake news stories are problematic not only for the credibility of online journalism, but also due to their detrimental real-world consequences. The primary research objective of this study is: What are the recent state-of-the-art modalities based on deep learning to detect fake news in social networks. This paper presents a systematic literature review of deep learning-based fake news detection models in social networks. The methodology followed a rigorous approach, including predefined criteria for study selection of deep learning modalities. This study focuses on the types of deep learning modalities; unimodal (refers to the use of a single model for analysis or modeling purposes) and multimodal models (refers to the integration of multiple models). The results of this review reveal the strengths and weaknesses of modalities approaches, as well as the limitations of low-resource languages datasets. Furthermore, it provides insights into future directions for deep learning models and different fact checking techniques. At the end of this study, we discuss the problem of fake news detection in the era of large language models in terms of advantages, drawbacks, and challenges.

PheNormGPT: a framework for extraction and normalization of key medical findings.

This manuscript presents PheNormGPT, a framework for extraction and normalization of key findings in clinical text. PheNormGPT relies on an innovative approach, leveraging large language models to extract key findings and phenotypic data in unstructured clinical text and map them to Human Phenotype Ontology concepts. It utilizes OpenAI's GPT-3.5 Turbo and GPT-4 models with fine-tuning and few-shot learning strategies, including a novel few-shot learning strategy for custom-tailored few-shot example selection per request. PheNormGPT was evaluated in the BioCreative VIII Track 3: Genetic Phenotype Extraction from Dysmorphology Physical Examination Entries shared task. PheNormGPT achieved an F1 score of 0.82 for standard matching and 0.72 for exact matching, securing first place for this shared task.

Mitigating adversarial manipulation in LLMs: a prompt-based approach to counter Jailbreak attacks (Prompt-G)

Large language models (LLMs) have become transformative tools in areas like text generation, natural language processing, and conversational AI. However, their widespread use introduces security risks, such as jailbreak attacks, which exploit LLM’s vulnerabilities to manipulate outputs or extract sensitive information. Malicious actors can use LLMs to spread misinformation, manipulate public opinion, and promote harmful ideologies, raising ethical concerns. Balancing safety and accuracy require carefully weighing potential risks against benefits. Prompt Guarding (Prompt-G) addresses these challenges by using vector databases and embedding techniques to assess the credibility of generated text, enabling real-time detection and filtering of malicious content. We collected and analyzed a dataset of Self Reminder attacks to identify and mitigate jailbreak attacks, ensuring that the LLM generates safe and accurate responses. In various attack scenarios, Prompt-G significantly reduced jailbreak success rates and effectively identified prompts that caused confusion or distraction in the LLM. Integrating our model with Llama 2 13B chat reduced the attack success rate (ASR) to 2.08%. The source code is available at: https://doi.org/10.5281/zenodo.13501821.

To What Extent Have LLMs Reshaped the Legal Domain So Far? A Scoping Literature Review

Understanding and explaining legal systems is very challenging due to their complex structure, specialized terminology, and multiple interpretations. Legal AI models are currently undergoing drastic advancements due to the development of Large Language Models (LLMs) that have achieved state-of-the-art performance on a wide range of tasks and are currently undergoing very rapid iterations. As an emerging field, the application of LLMs in the legal field is still in its early stages, with multiple challenges that need to be addressed. Our objective is to provide a comprehensive survey of legal LLMs, not only reviewing the models themselves but also analyzing their applications within the legal systems in different geographies. The paper begins by providing a high-level overview of AI technologies in the legal field and showcasing recent research advancements in LLMs, followed by practical implementations of legal LLMs. Two databases (i.e., SCOPUS and Web of Science) were considered alongside additional related studies that met our selection criteria. We used the PRISMA for Scoping Reviews (PRISMA-ScR) guidelines as the methodology to extract relevant studies and report our findings. The paper discusses and analyses the limitations and challenges faced by legal LLMs, including issues related to data, algorithms, and judicial practices. Moreover, we examine the extent to which such systems can be effectively deployed. The paper summarizes recommendations and future directions to address challenges, aiming to help stakeholders overcome limitations and integrate legal LLMs into the judicial system.

Large language model-based optical network log analysis using LLaMA2 with instruction tuning

The optical network encompasses numerous devices and links, generating a significant volume of logs. Analyzing these logs is significant for network optimization, failure diagnosis, and health monitoring. However, the large-scale and diverse formats of optical network logs present several challenges, including the high cost and difficulty of manual processing, insufficient semantic understanding in existing analysis methods, and the strict requirements for data security and privacy. Generative artificial intelligence (GAI) with powerful language understanding and generation capabilities has the potential to address these challenges. Large language models (LLMs) as a concrete realization of GAI are well-suited for analyzing DCI logs, replacing human experts and enhancing accuracy. Additionally, LLMs enable intelligent interactions with network administrators, automating tasks and improving operational efficiency. Moreover, fine-tuning with open-source LLMs protects data privacy and enhances log analysis accuracy. Therefore, we introduce LLMs and propose a log analysis method with instruction tuning using LLaMA2 for log parsing, anomaly detection and classification, anomaly analysis, and report generation. Real log data extracted from the field-deployed network was used to design and construct instruction tuning datasets. We utilized the dataset for instruction tuning and demonstrated and evaluated the effectiveness of the proposed scheme. The results indicate that this scheme improves the performance of log analysis tasks, especially a 14% improvement in exact match rate for log parsing, a 13% improvement in F1-score for anomaly detection and classification, and a 23% improvement in usability for anomaly analysis, compared with the best baselines.

Using large language models to create narrative events

Narratives play a crucial role in human communication, serving as a means to convey experiences, perspectives, and meanings across various domains. They are particularly significant in scientific communities, where narratives are often utilized to explain complex phenomena and share knowledge. This article explores the possibility of integrating large language models (LLMs) into a workflow that, exploiting the Semantic Web technologies, transforms raw textual data gathered by scientific communities into narratives. In particular, we focus on using LLMs to automatically create narrative events, maintaining the reliability of the generated texts. The study provides a conceptual definition of narrative events and evaluates the performance of different smaller LLMs compared to the requirements we identified. A key aspect of the experiment is the emphasis on maintaining the integrity of the original narratives in the LLM outputs, as experts often review texts produced by scientific communities to ensure their accuracy and reliability. We first perform an evaluation on a corpus of five narratives and then on a larger dataset comprising 124 narratives. LLaMA 2 is identified as the most suitable model for generating narrative events that closely align with the input texts, demonstrating its ability to generate high-quality narrative events. Prompt engineering techniques are then employed to enhance the performance of the selected model, leading to further improvements in the quality of the generated texts.

Application of large language models to intelligently analyze long construction contract texts

The emergence of large language models (LLMs) has provided an opportunity to assist humans in quickly reading, searching, and understanding the contents of construction contracts. However, the limited context length of LLMs restricts their ability to process lengthy contract documents, which hinders their application in the construction industry. This study proposes intelligent analysis methods for long construction contracts, which enables LLMs to handle contracts that exceed their context length through a two-stage text segmentation. We further utilized the segmented text chunks for content compression and intelligent question-answering applications. The FIDIC contract was used for testing. A condensed version of lengthy contracts summarizes the provisions in a shorter format. It maintains a high level of correctness and readability, offering practitioners additional options to read contracts of varying lengths based on their needs. The satisfaction rate of the question-answering outcomes reached 93.3%, allowing practitioners to quickly obtain specific clauses of interest and relevant contract knowledge through personalized queries. The performance of the 8K model using our methods is comparable to that of the 128K long-text models, while reducing computational power. This study expands the potential for applications of LLMs in contract management within the construction industry.

Descriptive Case Analysis on the Application of Prompt Engineering in Business Management

In recent years, the application of AI has been witnessed in various domains and disciplines, mainly AI in terms of large-scale pre-trained language models (LMs) and Visual Language models which are the prompt engineering types that have gained increasing attention in the field of Business and Management field. The prompt engineering approach is considered vital leverage in the areas of BM tasks. In this AI-generated review, we will try to discuss various aspects of prompt engineering encompassing the domain of management and also highlight various applications in this area. This paper also tries to examine the challenges of applying prompt engineering in the context of business management. This is a descriptive case study research exploring various facets of prompt engineering and application on the spectrum of business management. Through this exploration, this paper intends to contribute to the evolving dialogue on the integration of human creativity with generative AI capabilities, offering insights into the future of effective and ethical AI interaction. Prompt engineering, Artificial intelligence, large language models, Visual language models, Business Management, Human-AI interaction, Chat GPT, Generative AI

Improving Workplace Well-being in Modern Organizations: A Review of Large Language Model-based Mental Health Chatbots

The global rise in mental disorders, particularly in workplaces, necessitated innovative and scalable solutions for delivering therapy. Large Language Model (LLM)-based mental health chatbots have rapidly emerged as a promising tool for overcoming the time, cost, and accessibility constraints often associated with traditional mental health therapy. However, LLM-based mental health chatbots are in their nascency, with significant opportunities to enhance their capabilities to operate within organizational contexts. To this end, this research seeks to examine the role and development of LLMs in mental health chatbots over the past half-decade. Through our review, we identified over 50 mental health-related chatbots, including 22 LLM-based models targeting general mental health, depression, anxiety, stress, and suicide ideation. These chatbots are primarily used for emotional support and guidance but often lack capabilities specifically designed for workplace mental health, where such issues are increasingly prevalent. The review covers their development, applications, evaluation, ethical concerns, integration with traditional services, LLM-as-a-service, and various other business implications in organizational settings. We provide a research illustration of how LLM-based approaches could overcome the identified limitations and also offer a system that could help facilitate systematic evaluation of LLM-based mental health chatbots. We offer suggestions for future research tailored to workplace mental health needs.

Foundation models assist in human–robot collaboration assembly

Human–robot collaboration (HRC) is a novel manufacturing paradigm designed to fully leverage the advantage of humans and robots, efficiently and flexibly accomplishing customized manufacturing tasks. However, existing HRC systems lack the transfer and generalization capability for environment perception and task reasoning. These limitations manifest in: (1) current methods rely on specialized models to perceive scenes; and need retraining the model when facing unseen objects. (2) current methods only address predefined tasks, and cannot support undefined task reasoning. To avoid these limitations, this paper proposes a novel HRC approach based on Foundation Models (FMs), including Large Language models (LLMs) and Vision Foundation Models (VFMs). Specifically, a LLMs-based task reasoning method is introduced, utilizing prompt learning to transfer LLMs into the domain of HRC tasks, supporting undefined task reasoning. A VFMs-based scene semantic perception method is proposed, integrating various VFMs to achieve scene perception without training. Finally, a FMs-based HRC system is developed, comprising perception, reasoning, and execution modules for more flexible and generalized HRC. The superior performances of FMs in perception and reasoning are demonstrated by extensive experiments. Furthermore, the feasibility and effectiveness of the FMs-based HRC system are validated through an part assembly case involving a satellite component model.

The Use of Large Language Models Such As ChatGPT on Delivering Patient Information Relating to Surgery.

not applicable.

The Energy Effect of Blockchain Technology Innovation in the Industry 5.0 Era--From the Perspective of Carbon Emissions

In the era of Industry 5.0, innovation in blockchain technology is expected to have a significant impact on energy efficiency and carbon emissions, especially in hybrid energy systems. This paper has obtained the innovation data on the blockchain patent of listed companies in China from 2010 to 2022 by machine learning and text analysis methodology of large language model. On this basis, the relationship between the enterprises’ blockchain technology innovation and the intensity of fossil energy consumption has been discussed. The study found that (1) the enterprises’ blockchain technology innovation is able to reduce the intensity of fossil energy consumption, and this conclusion remains valid via controlling for endogeneity and upon the completion of robustness test. (2) In terms of the mechanism, the enterprises’ blockchain technology innovation can achieve energy transition by reducing the cost of internal control, strengthening external supervision, improving the efficiency of resource allocation and strengthening the channels of supply chain management. (3) Further research has found that the energy effects as a result of blockchain technology innovation help to improve operational efficiency and lower the level of carbon emission. Our research provides scientific support for breaking the "technological energy paradox" in the field of energy economy while providing empirical evidence of "blockchain carbon reduction" for technologies such as blockchain BaaS alliance, cross-operator network spectrum, and shared base stations that will be applied on a large scale in the Industry 5.0 Era.

Evaluating the Effectiveness of a Large Language Model in the Psychological Assessment of Potential Liver Transplant Candidates: A Feasibility Study

Introduction: The use of artificial intelligence (AI) in medical settings has shown promise in various domains including diagnostics, treatment recommendation, and patient management. Recent advances have explored the potential of AI in pre-surgical assessments, but its application in transplant psychology remains unexplored. The objective of this study is to assess the effectiveness of AI in the context of screening potential liver transplant donors and recipients. To assess the feasibility of using ChatGPT-4 to screen potential liver transplant donors and recipients. Methods: This study utilizes a cross-sectional research design to evaluate the feasibility of using ChatGPT-4 in the preliminary screening of living liver donors and liver transplant candidates. The study aims to determine the accuracy and reliability of ChatGPT-4 in assessing hypothetical scenarios involving potential donors and recipients. Results: The analysis showed no found no significant differences between ChatGPT-4 and the expert panel in assessing liver transplant candidates, demonstrating an overall accuracy of 83.58%, sensitivity of 56.10%, and specificity of 80.49%. Additionally, the Cohen’s Kappa statistic of 0.68 (95% CI: 0.52-0.83) indicated substantial agreement between ChatGPT-4 and the psychologists’ evaluations. The absence of false positives (0%) and a low false negative rate (8%) emphasize ChatGPT-4's cautious and accurate decision-making capabilities. Conclusion: The findings of this study demonstrate that ChatGPT-4 has the potential to serve as an effective screening tool for liver transplant candidates, complementing the work of human experts and enhancing the overall efficiency of the transplant process. While challenges remain, the integration of AI into the liver transplantation workflow could lead to significant improvements in candidate evaluation and patient outcomes, paving the way for the broader application of AI in clinical practice.

Enhancing Chinese comprehension and reasoning for large language models: an efficient LoRA fine-tuning and tree of thoughts framework

Enhancing Chinese comprehension and reasoning for large language models: an efficient LoRA fine-tuning and tree of thoughts framework