Natural Language Responses Research Articles

Revolutionizing Customer Service : The Impact of Large Language Models on Chatbot Performance

This article examines the transformative impact of Large Language Models (LLMs) on customer service chatbots across various industries. Through a comprehensive analysis of case studies, the article explores the implementation strategies, performance metrics, and technological advantages of LLM-powered chatbots in sectors including e-commerce, telecommunications, financial services, healthcare, and travel. The article highlights the advanced capabilities of LLMs in natural language understanding, response generation, and continuous learning, demonstrating their potential to significantly enhance customer interactions and operational efficiency. The article also addresses key technical challenges in implementing LLM chatbots, such as data privacy, response accuracy, and scalability, offering potential solutions for each. Finally, it discusses future directions for LLM-powered chatbots, including multimodal capabilities, emotion recognition, and predictive customer service, providing insights into the evolving landscape of AI-driven customer engagement.

Virtual Assistant by using Large Language Model

Abstract— This paper presents the development of a virtual assistant named “Pragyan,” utilizing a Large Language Model (LLM) for natural language understanding and response generation. The virtual assistant is designed to interact with users through text and speech inputs, providing informative and conversational responses. Pragyan is built with a frontend user interface, a backend server, and a database for efficient communication and data management. The performance evaluation and result analysis demonstrate the effectiveness and user-friendliness of the virtual assistant Keywords: Virtual Assistant, Large Language Model, Natural Language Processing, Frontend, Backend, Database, Performance Evaluation, Result Analysis.

Intent Recognition and slot filling for ecommerce chatbot

With the continuous evolution of natural language processing (NLP), conversational agents have become pivotal in enhancing user engagement and satisfaction in the e-commerce domain. This paper presents a comprehensive study and implementation of intent recognition and slot filling techniques tailored for an e-commerce chatbot. Leveraging a dataset of 8,000 samples, we employed a Convolutional Neural Network (CNN) combined with Long Short-Term Memory (LSTM) networks for intent recognition, achieving an accuracy of 87%. For slot filling, we utilized the Bidirectional Encoder Representations from Transformers (BERT) model, also attaining an accuracy of 87%. The chatbot seamlessly integrates with an e-commerce database, using OpenAI's GPT-3 to generate natural language responses from query results. Our system demonstrates significant advancements in processing user queries, generating precise database queries, and providing coherent and relevant responses, thereby enhancing the overall user experience. This work aims to equip researchers and practitioners with insights into the methodologies and challenges in developing sophisticated e-commerce chatbots, fostering further innovation in this field. Key Words: Intent Recognition, Slot Filling, Convolutional Neural Network (CNN), Long Short-Term Memory (LSTM), Bidirectional Encoder Representations from Transformers (BERT), GPT-3, Natural Language Understanding (NLU), Chatbot

My AI students: Evaluating the proficiency of three AI chatbots in <i>completeness</i> and <i>accuracy</i>

A new era of artificial intelligence (AI) has begun, which can radically alter how humans interact with and profit from technology. The confluence of chat interfaces with large language models lets humans write a natural language inquiry and receive a natural language response from a machine. This experimental design study tests the capabilities of three popular AI chatbot services referred to as my AI students: Microsoft Bing, Google Bard, and OpenAI ChatGPT on <i>completeness</i> and <i>accuracy</i>. A Likert scale was used to rate c<i>ompleteness </i>and <i>accuracy,</i> respectively, a three-point and five-point. Descriptive statistics and non-parametric tests were used to compare marks and scale ratings. The results show that AI chatbots were awarded a score of 80.0% overall. However, they struggled with answering questions from the higher Bloom’s taxonomic levels. The median <i>completeness</i> was 3.00 with a mean of 2.75 and the median <i>accuracy</i> was 5.00 with a mean of 4.48 across all Bloom’s taxonomy questions (n=128). Overall, the<i> completeness</i> of the solution was rated mostly incomplete due to limited response (76.2%), while <i>accuracy</i> was rated mostly correct (83.3%). In some cases, generative text was found to be verbose and disembodied, lacking perspective and coherency. Microsoft Bing ranked first among the three AI text generative tools in providing correct answers (92.0%). The Kruskal-Wallis test revealed a significant difference in <i>completeness </i>(asymp. sig.=0.037, p<0.05) and <i>accuracy</i> (asymp. sig.=0.006, p<0.05) among the three AI chatbots. A series of Mann and Whitney tests were carried out showing no significance between AI chatbots for <i>completeness</i> (all p-values>0.015 and 0<r<0.2), while a significant difference was found for <i>accuracy</i> between Google Bard and Microsoft Bing (asymp. sig.=0.002, p<0.05, r=0.3 medium effect). The findings suggest that while AI chatbots can generate comprehensive and correct responses, they may have limits when dealing with more complicated cognitive tasks.

HOMMIE: CONVERSATIONAL AI ASSISTANT

Conversational AI assistants have witnessed remarkable progress in recent years, transforming human-computer interaction across a spectrum of applications. This paper offers a comprehensive overview of the state-of-the-art techniques, methodologies, and challenges in the field. We examine the core components of conversational AI, including natural language understanding, dialogue management, and response generation. Additionally, we address key challenges such as context modeling, personalization, and ethical considerations. The paper serves as a roadmap for researchers and developers, highlighting current achievements and avenues for future advancements in the dynamic domain of conversational AI. keyword - Conversational AI, NLP, Assistant, Machine Learning, Chatbot

Career Recognition & Academic Counseling (CRAC Bot)

In recent years, the advancement of Artificial Intelligence (AI) technologies has revolutionized various sectors, including education and career guidance. This research paper presents an innovative approach to integrating Career Recognition and Academic Counselling Chatbots with College and Learning Management Systems (LLMs) using Generative AI and Large Language Models (LLMs). The proposed system aims to enhance the academic counselling experience for students by providing personalized career guidance, course recommendations, and academic support through an intelligent chatbot interface. Leveraging Generative AI techniques, the chatbot can generate natural language responses and engage in meaningful conversations with users, facilitating efficient communication and knowledge dissemination. Additionally, the integration with College and LLMS enables seamless access to academic resources, course materials, and learning opportunities, empowering students to make informed decisions about their academic and career pathways. The research paper discusses the design, development, and implementation of the integrated system, along with a comprehensive evaluation of its effectiveness in enhancing student engagement, academic success, and career readiness. Through empirical studies and user feedback analysis, the research demonstrates the potential impact of AI-driven academic counselling solutions in improving student outcomes and fostering a supportive learning environment. Finally, the paper discusses the implications, challenges, and future directions of integrating Generative AI and LLMs into academic counselling and career recognition systems, highlighting opportunities for further research and innovation in this field. Keywords: Generative AI, LLM, Chatbot, AI-learning Management systems, academic counselling

Enhancing Banking Chatbot Experience through Hybrid Conversational-Generative AI Approaches

In today's digital age, chatbots have become an integral part of the banking industry, offering customers quick and efficient assistance. However, to meet the growing demands of customers and provide a seamless user experience, banking chatbots are evolving beyond rule-based systems. Traditional rule-based chatbots often fall short in handling complex queries and delivering personalized responses. This research proposal outlines a study aimed at enhancing the banking chatbot experience through the hybrid conversational-generative AI approaches. The primary focus of this research is to combine the strengths of conversational AI, which excels in understanding and generating natural language responses, with generative AI, capable of generating creative and context-aware responses. By merging these two AI paradigms, banking chatbots can provide more human-like interactions, understand complex queries, and offer personalized responses

What is the impact of artificial intelligence-based chatbots on infodemic management?

Artificial intelligence (AI) chatbots have the potential to revolutionize online health information-seeking behavior by delivering up-to-date information on a wide range of health topics. They generate personalized responses to user queries through their ability to process extensive amounts of text, analyze trends, and generate natural language responses. Chatbots can manage infodemic by debunking online health misinformation on a large scale. Nevertheless, system accuracy remains technically challenging. Chatbots require training on diverse and representative datasets, security to protect against malicious actors, and updates to keep up-to-date on scientific progress. Therefore, although AI chatbots hold significant potential in assisting infodemic management, it is essential to approach their outputs with caution due to their current limitations.

Occupation classification model based on DistilKoBERT: using the 5th and 6th Korean Working Condition Surveys.

Accurate occupation classification is essential in various fields, including policy development and epidemiological studies. This study aims to develop an occupation classification model based on DistilKoBERT. This study used data from the 5th and 6th Korean Working Conditions Surveys conducted in 2017 and 2020, respectively. A total of 99,665 survey participants, who were nationally representative of Korean workers, were included. We used natural language responses regarding their job responsibilities and occupational codes based on the Korean Standard Classification of Occupations (7th version, 3-digit codes). The dataset was randomly split into training and test datasets in a ratio of 7:3. The occupation classification model based on DistilKoBERT was fine-tuned using the training dataset, and the model was evaluated using the test dataset. The accuracy, precision, recall, and F1 score were calculated as evaluation metrics. The final model, which classified 28,996 survey participants in the test dataset into 142 occupational codes, exhibited an accuracy of 84.44%. For the evaluation metrics, the precision, recall, and F1 score of the model, calculated by weighting based on the sample size, were 0.83, 0.84, and 0.83, respectively. The model demonstrated high precision in the classification of service and sales workers yet exhibited low precision in the classification of managers. In addition, it displayed high precision in classifying occupations prominently represented in the training dataset. This study developed an occupation classification system based on DistilKoBERT, which demonstrated reasonable performance. Despite further efforts to enhance the classification accuracy, this automated occupation classification model holds promise for advancing epidemiological studies in the fields of occupational safety and health.

Harnessing Large Language Models in Medical Research and Scientific Writing: A Closer Look to The Future

Large Language Models (LLMs), a form of artificial intelligence generating natural language responses based on user input, have demonstrated potential across various applications such as entertainment, education, and customer service. This review comprehensively highlights their current research status and potential applications within the medical domain, addressing the challenges and opportunities for future development and implementation. Key aspects covered include diverse data sources for training and testing, such as electronic health records and clinical trials; ethical considerations, including privacy and consent; evaluation techniques focusing on accuracy and coherence; and clinical applications ranging from diagnosis to patient education. The review concludes that LLMs hold significant promise for enhancing the quality and efficiency of medical research and scientific writing but also emphasize the need for careful design and regulation to ensure safety and reliability.

Evaluating the performance of ChatGPT in writing autopsy clinicopathological correlations

Abstract ChatGPT is a chatbot launched by OpenAI in November 2022 that has generated excitement due to its utility in several technical fields including medicine. The application has gained media attention due to concerns of academic plagiarism, including its use by students for writing projects. In scientific writing, a preprint in bioarxiv reports human reviewers failing to identify a portion of ChatGPT generated abstracts. Several articles published in early 2023 list ChatGPT as a coauthor, generating debate. In medicine, Gilson et al found ChatGPT crossed the 60% threshold for the NBME Free Step-1 dataset. Patel and Lam wrote a commentary examining the value of ChatGPT in writing discharge summaries. A literature review in PubMed yielded articles further examining the utility of ChatGPT in scientific writing, however none looking at its use in pathology report writing was found. To examine this, we investigated the capabilities of ChatGPT in writing autopsy clinicopathologic reports. The February 13 version of ChatGPT was used to generate autopsy clinicopathologic correlations using prompts with the same patient demographic, same medical history of hypertension and coronary disease, and varying autopsy findings. These findings include cardiac infarct, saddle emboli, basilar artery thrombus, liver cyst, and leukoplakia. A prompt with no significant autopsy findings was also included. The prompts note a lack of imaging, labs, and toxicology. The generated responses were examined and compared. ChatGPT generated multi-sentence responses incorporating patient information, history, and major autopsy findings per prompt, ranging in length from 8 to 14 sentences. Each output varied in style, with some responses including sections for gross and microscopic findings, while others only consisting of clinicopathologic correlation. Each response was generated between 42 seconds to 1 minute. The application generated plausible correlations between the medical histories and autopsy findings. ChatGPT correlated the findings to the patient’s supplied history in prompts with a cardiovascular autopsy finding. For the liver cyst, ChatGPT linked cyst formation to portal hypertension. For leukoplakia, ChatGPT suggested the presence of an upper GI cancer with cardiovascular history also likely contributing to death. For the prompt without significant findings, ChatGPT noted that the autopsy was inconclusive. ChatGPT debuted with an increasing amount of media attention due to its capability of producing natural language responses and surprising knowledge within multiple technical fields. ChatGPT is currently being used in academic writing at multiple levels within multiple fields. In medicine, ChatGPT has done well in multiple USMLE question banks. This study found that ChatGPT generated coherent clinicopathologic correlations commensurate to the amount of information supplied in the initial prompt. While useful as an initial framework, human review is still necessary to refine the response to make a more coherent report and to review made-up findings and dubious correlations.

A Structured Narrative Prompt for Prompting Narratives from Large Language Models: Sentiment Assessment of ChatGPT-Generated Narratives and Real Tweets

Large language models (LLMs) excel in providing natural language responses that sound authoritative, reflect knowledge of the context area, and can present from a range of varied perspectives. Agent-based models and simulations consist of simulated agents that interact within a simulated environment to explore societal, social, and ethical, among other, problems. Simulated agents generate large volumes of data and discerning useful and relevant content is an onerous task. LLMs can help in communicating agents’ perspectives on key life events by providing natural language narratives. However, these narratives should be factual, transparent, and reproducible. Therefore, we present a structured narrative prompt for sending queries to LLMs, we experiment with the narrative generation process using OpenAI’s ChatGPT, and we assess statistically significant differences across 11 Positive and Negative Affect Schedule (PANAS) sentiment levels between the generated narratives and real tweets using chi-squared tests and Fisher’s exact tests. The narrative prompt structure effectively yields narratives with the desired components from ChatGPT. In four out of forty-four categories, ChatGPT generated narratives which have sentiment scores that were not discernibly different, in terms of statistical significance (alpha level α=0.05), from the sentiment expressed in real tweets. Three outcomes are provided: (1) a list of benefits and challenges for LLMs in narrative generation; (2) a structured prompt for requesting narratives of an LLM chatbot based on simulated agents’ information; (3) an assessment of statistical significance in the sentiment prevalence of the generated narratives compared to real tweets. This indicates significant promise in the utilization of LLMs for helping to connect a simulated agent’s experiences with real people.

Dynamic prompt-based virtual assistant framework for BIM information search

Efficient information search from building information models (BIMs) requires deep BIM knowledge or extensive engineering efforts for building natural language (NL)-based interfaces. To address this challenge, this paper introduces a dynamic prompt-based virtual assistant framework dubbed “BIMS-GPT” that integrates generative pre-trained transformer (GPT) technologies, supporting NL-based BIM search. To understand users' NL queries, extract relevant information from BIM databases, and deliver NL responses along with 3D visualizations, a dynamic prompt-based process was developed. In a case study, BIMS-GPT's functionality is demonstrated through a virtual assistant prototype for a hospital building. When evaluated with a BIM query dataset, the approach achieves accuracy rates of 99.5% for classifying NL queries with incorporating 2% of the data in prompts. This paper contributes to the advancement of effective and versatile virtual assistants for BIMs in the construction industry as it significantly enhances BIM accessibility while reducing the engineering and training data prerequisites for processing NL queries.

Ontology-driven development of dialogue systems

A new technique and its software implementation are presented to create a deeply semantically structured ontology using plain natural language text as input, without regular structure or any previous tagging and markup. The new approach is primarily aimed at highly inflectional languages, and is implemented for Ukrainian. The automatically created ontologies (in OWL) could be easily converted to other graph databases formats, such as Neo4j, and were successfully evaluated as valid ontologies using Protégé, RDFlib and Neo4j environments. An integrated approach is proposed for the development of natural language dialogue systems driven by the ontologyrelated graph database using the Cypher language for the formal queries. The original phrases are subject to a special method of semantic analysis, which determines the type of formal query to the database. The essence of the analysis is that the text of the user’s phrase goes through a series of checks. Based on their results, a set of basic templates for the formal requests are determined, as well as additional constructions that are attached to the basic template. Some of the checks may also return the notion of substitution to certain specified positions of the formal query. Formal queries can return both contexts and lists of ontology concepts. In addition to concepts, queries can also return information about specific semantic predicates that connect them, which simplifies the synthesis of natural language responses. The synthesis of answers is based on special templates, the choice of which is directly related to the corresponding template of the formal query.

The applications of chatbot

A chatbot is a program that uses natural language to simulate a human conversation. and can receive questions from users to answer, as well as engage in casual conversation with users in a set format or help them complete a matter. It can free up personnel in an organization that is needed for reasonably stable, repetitive, and demanding tasks or procedures. Natural language processing (NLP), an essential aspect of artificial intelligence, forms the foundation of the system., and neural networks (NNs) that receive natural language and process responses. In recent years, chatbots have grown significantly with the rise of artificial intelligence, with achievements by major Internet companies, various universities, and research institutes. This review provides a detailed study of chatbots in five areas of intelligence in recent years, including education, intelligent question and answer, customer service, entertainment, and personal assistants. The goal is to summarize the cutting-edge and effective technologies in each area. In reviewing these papers, we pay closer attention to the technology in this paper's potential development possibilities and whether relevant experiments or detailed descriptions are done for the proposed future extensions, etc. While we discuss and study the current technologies, we also pay attention to the possibilities and trends in the development of new fields.

A Post-Processing Framework for Crowd Worker Responses Using Large Language Models

To develop quality crowdsourcing systems, aggregating responses from workers is a critical issue. However, it has been difficult to construct an automatic mechanism that flexibly aggregates worker responses in natural language. Accordingly, responses need to be collected in a standardized format, such as binary-choice or multiple categorizations, to avoid large aggregation costs. Recently, with the advent of large language models (LLMs), natural language responses can be automatically and flexibly aggregated. We propose a framework that uses LLMs to flexibly aggregate natural language responses from workers and, as a promising example, consider this framework for crime detection from surveillance cameras using crowdsourced cognitive abilities. In an experiment using subjective evaluation, our proposed framework is shown to be effective for automatically aggregating natural language responses from crowd workers.

Mani-GPT: A Generative Model for Interactive Robotic Manipulation

In real-world scenarios, human dialogues are multi-round and diverse. Furthermore, human instructions can be unclear and human responses are unrestricted. Interactive robots face difficulties in understanding human intents and generating suitable strategies for assisting individuals through manipulation. In this article, we propose Mani-GPT, a Generative Pre-trained Transformer (GPT) for interactive robotic manipulation. The proposed model has the ability to understand the environment through object information, understand human intent through dialogues, generate natural language responses to human input, and generate appropriate manipulation plans to assist the human. This makes the human-robot interaction more natural and humanized. In our experiment, Mani-GPT outperforms existing algorithms with an accuracy of 84.6% in intent recognition and decision-making for actions. Furthermore, it demonstrates satisfying performance in real-world dialogue tests with users, achieving an average response accuracy of 70%.

A dialogue system based on ontology automatically built through a natural language text analysis

An integrated approach is created to the development of natural-language dialogue systems driven by an ontological graph database. Ontology here has a defined regular structure that contains typed semantic relationships between concepts, as well as related contexts, which may also have a multilevel structure and additional typing. The ontology is created automatically due to the semantic analysis of a natural language using a specially developed original software, which is set up to work with inflected languages, in particular Ukrainian. The ontology description is serialized in OWL format. To work as part of the dialog system, the ontology is transferred to the graph database Neo4j. The Cypher language is used for formal queries. The original phrases of the user are subject to a special method of semantic analysis, which determines the type of formal query to the database. The essence of the analysis is that the text of the user phrase goes through a series of checks. Based on their results, a set of basic templates for formal requests is determined, as well as additional constructions that are attached to the basic template. Some of the checks may also return the notion of substitution to certain specified positions of the formal query. Formal queries can return both contexts and lists of ontology concepts. In addition to concepts, queries can also return information about specific semantic predicates that connect them, which simplifies the synthesis of natural language responses. The synthesis of answers is based on special templates, the choice of which is directly related to the corresponding template of the formal query.

VISUAL QUESTIONING AND ANSWERING

In general, a VQA system is an algorithm that takes a picture and natural language query about the image as input and produces natural language response as output. The nature of a multi-discipline research problem necessitates this. This is a diagnostic dataset that assesses a variety of visual reason skills. VQA features few biases and through annotations defining the type of reasoning required for each question. The dataset is used to examine a number of recent visual reasoning systems, revealing new information about their capabilities and limitations.

Meta-evaluation of Conversational Search Evaluation Metrics

Conversational search systems, such as Google assistant and Microsoft Cortana, enable users to interact with search systems in multiple rounds through natural language dialogues. Evaluating such systems is very challenging, given that any natural language responses could be generated, and users commonly interact for multiple semantically coherent rounds to accomplish a search task. Although prior studies proposed many evaluation metrics, the extent of how those measures effectively capture user preference remain to be investigated. In this article, we systematically meta-evaluate a variety of conversational search metrics. We specifically study three perspectives on those metrics: (1) reliability : the ability to detect “actual” performance differences as opposed to those observed by chance; (2) fidelity : the ability to agree with ultimate user preference; and (3) intuitiveness : the ability to capture any property deemed important: adequacy, informativeness, and fluency in the context of conversational search. By conducting experiments on two test collections, we find that the performance of different metrics vary significantly across different scenarios, whereas consistent with prior studies, existing metrics only achieve weak correlation with ultimate user preference and satisfaction. METEOR is, comparatively speaking, the best existing single-turn metric considering all three perspectives. We also demonstrate that adapted session-based evaluation metrics can be used to measure multi-turn conversational search, achieving moderate concordance with user satisfaction. To our knowledge, our work establishes the most comprehensive meta-evaluation for conversational search to date.