Chatbot Research Articles

An Investigation into Students' Perception of Using Call Annie as a Virtual Conversational Partner

There is no doubt that using AI tools to learn a foreign language has become more common in recent years. This study aims to explore students’ perception of using Call Annie – an AI-powered tool – to practice speaking regularly, as part of their homework to find out their perceived advantages and disadvantages after conversing with the application. This research project employed 85 English majors at a university for the investigation, in which both quantitative and qualitative data were collected via questionnaires and interviews to observe their interactions with the Call Annie chatbot for homework assignments as conversational partners for three weeks. After the period, students were given surveys to record their perception, and after their final speaking test, a focus group interview was held with 3 students. Findings show students’ willingness and confidence to utilize Call Annie in the future for their speaking skills, as well as their positive attitudes. However, the data also shows that students were worried about using the tool due to concerns about authenticity and language content. In the future, AI chatbots may be used as conversational partners, which could enable teachers to take more active roles in individualized approaches to teaching speaking.

Comparative analysis of ChatGPT and Gemini (Bard) in medical inquiry: a scoping review

IntroductionArtificial intelligence and machine learning are popular interconnected technologies. AI chatbots like ChatGPT and Gemini show considerable promise in medical inquiries. This scoping review aims to assess the accuracy and response length (in characters) of ChatGPT and Gemini in medical applications.MethodsThe eligible databases were searched to find studies published in English from January 1 to October 20, 2023. The inclusion criteria consisted of studies that focused on using AI in medicine and assessed outcomes based on the accuracy and character count (length) of ChatGPT and Gemini. Data collected from the studies included the first author's name, the country where the study was conducted, the type of study design, publication year, sample size, medical speciality, and the accuracy and response length.ResultsThe initial search identified 64 papers, with 11 meeting the inclusion criteria, involving 1,177 samples. ChatGPT showed higher accuracy in radiology (87.43% vs. Gemini's 71%) and shorter responses (907 vs. 1,428 characters). Similar trends were noted in other specialties. However, Gemini outperformed ChatGPT in emergency scenarios (87% vs. 77%) and in renal diets with low potassium and high phosphorus (79% vs. 60% and 100% vs. 77%). Statistical analysis confirms that ChatGPT has greater accuracy and shorter responses than Gemini in medical studies, with a p-value of <.001 for both metrics.ConclusionThis Scoping review suggests that ChatGPT may demonstrate higher accuracy and provide shorter responses than Gemini in medical studies.

Anthropomorphic generative AI chatbots for enhancing customer engagement, experience and recommendation

Anthropomorphic generative AI chatbots for enhancing customer engagement, experience and recommendation

Sentiment and Emotion Modeling in Text-based Conversations utilizing ChatGPT

Emotional Intelligence (EI) constitutes a vital element of human communication, and its integration into text-based dialogues has gained great significance in the modern digital era. The present paper proposes an innovative method for modeling sentiment and emotion within text-based conversations using the ChatGPT language model. The advancements in sentiment and emotion recognition are centered on the role of EI in text-based conversational models. The study underscores the significance of diverse datasets, including Interactive Emotional Dyadic Motion Capture (IEMOCAP), MELD, EMORYNLP, and DAILYDIALOG, for training and evaluating emotion detection algorithms. IEMOCAP and MELD offer detailed emotional annotations, EMORYNLP emphasizes sensitive dialogue scenarios, and DAILYDIALOG encompasses a wide range of everyday interactions, providing distinct advantages for capturing emotional subtleties. The proficiency of different emotion categorization models, including ChatGPT and models with four levels of detail, is demonstrated through their capacity to understand and respond to emotions aptly. The crucial role of conversational AI with sophisticated EI in fostering empathy and context-sensitive interactions is emphasized.

Digital transformation of relocated higher education institutions in Ukraine under martial law

The ongoing Russia-Ukraine war has profoundly disrupted the higher education landscape, compelling numerous institutions to adapt to unprecedented challenges. This study investigates the resilience and adaptive strategies of relocated higher education institutions under martial law, focusing on Berdyansk State Pedagogical University. The analysis emphasizes the critical role of digital transformation in sustaining academic operations amidst displacement. Methodologically, the study integrates qualitative interviews and quantitative analysis, exploring how cloud technologies, learning management systems, and AI-driven chatbots contributed to continuity in education. The results reveal that digital platforms ensured accessibility to educational resources, increased student engagement, and enhanced institutional resilience. Over 85% of surveyed participants identified learning management systems’ platforms as pivotal in maintaining educational quality, while AI chatbots were instrumental during crises, offering real-time communication and support even during power outages. Additionally, cloud-based solutions enabled the preservation of critical data and ensured uninterrupted access to academic resources, facilitating smooth transitions for both faculty and students. The findings underline that digital transformation not only mitigates immediate disruptions but also fosters long-term innovation in higher education institutions operating in war zones. This study offers valuable insights into how relocated institutions can leverage digital tools to build resilience, sustain educational quality, and adapt to evolving challenges in war-affected regions. AcknowledgmentsWe are grateful to the Armed Forces of Ukraine for allowing us to engage in scientific research. This work was supported by the project “Supporting the cooperation of the University of Opole with Ukrainian universities within the FORTHEM Alliance 2024.”

Man versus machine: Comparative analysis of ChatGPT’s colon cancer guidance and expert opinion.

299 Background: AI Chatbots, such as ChatGPT, are increasingly being utilized across various aspects of society, including healthcare. Despite this growing usage, their role within the field of oncology remains underdeveloped. The objective of this study is to evaluate ChatGPT’s ability to accurately respond to patient inquiries regarding colon cancer by comparing its responses to assessments from expert clinical oncologists. Methods: Ten comprehensive questions were compiled by reviewing commonly asked questions from reputable sources, including the American Society of Colon & Rectal Surgeons, Mount Sinai, the National Cancer Institute, Mayo Clinic, and the American Cancer Society. The questions were categorized into two categories based on their content: General Oncology Characteristics (covering symptoms, screening, and prevention) and for Diagnosis & Treatment. These questions were then entered into ChatGPT, with prompts designed to simulate patient inquiries. The AI-generated responses were subsequently evaluated by oncology experts using a 5-point Likert scale to assess their accuracy and relevance, with scores reflecting the experts' level of agreement with the answers. Results: On a five-point Likert scale, with 1 representing "strongly disagree" and 5 representing "strongly agree," the mean score was 4.72. ANOVA analysis was performed and there was no statistically significant difference in the mean score across all raters (p = 0.221). However, ratings between the two categories were statistically significant (p=0.034). Conclusions: This study demonstrates that ChatGPT can provide accurate and relevant responses to patient inquiries about colon cancer, as assessed by medical oncology experts. With an average rating of 4.72 on a 5-point Likert scale, ChatGPT’s responses closely align with expert opinion, particularly for general characteristics category which included symptoms, prevention, and screening. However, responses related to diagnosis and treatment showed a statistically significant difference between expert opinions and AI, indicating that the experts agreed less with this component of AI's responses. These findings highlight the potential of AI chatbots in supplementing patient education in oncology, though further research is necessary to explore its limitations and expand its clinical utilities.

Generative AI Chatbots for Reliable Cancer Information: Evaluating web-search, multilingual, and reference capabilities of emerging large language models

Generative AI Chatbots for Reliable Cancer Information: Evaluating web-search, multilingual, and reference capabilities of emerging large language models

Proposing the “Digital Agenticity Theory” to analyze user engagement in conversational AI chatbot

Proposing the “Digital Agenticity Theory” to analyze user engagement in conversational AI chatbot

Generative artificial intelligence in hospitality and tourism: future capabilities, AI prompts and real-world applications

ABSTRACT The conversational AI emergence technologies like ChatGPT, Claude AI, and Gemini AI are transforming the hospitality and tourism industries. While existing research explores theoretical applications, a critical gap exists in understanding real-world implementation challenges, prompting strategies, and operational impacts of ChatGPT and other AI applications. This study investigates AI integration patterns, prompt optimization techniques (e.g. Input-Output, Chain-of-Thought, and Tree-of-Thought approach), and implementation strategies through a mixed-methods approach. We conducted a bibliometric analysis of 80 studies using R-Biblioshiny, followed by surveys and interviews with stakeholders, including guests/travelers, managers, and academics. Our empirical analysis measures personalization, user satisfaction, operational efficiency, and research productivity. The results demonstrate that while generative AI can enhance customer experiences and operational efficiency, success depends on selecting appropriate prompting methodologies, ethical considerations, workforce development, and community impact. This research provides an integrated framework for responsible AI adoption while identifying critical areas for future research.

Reliability of AI Chatbots in Categorising Foods by Oxalate Content /Yapay Zeka Araçlarının Gıdaları Oksalat İçeriğine Göre Sınıflandırmadaki Güvenilirliği

Aim: AI chatbots have shown promise in food classification tasks, but their accuracy in categorising foods based on specific nutritional content, such as oxalates, has not been thoroughly evaluated in the Turkish language. This study assesses the performance of three AI chatbots—ChatGPT 4.0, Gemini, and Microsoft Copilot—in classifying foods according to their oxalate content. Materials and Methods: A dataset of 63 diverse food items, including commonly consumed Turkish foods, was used to evaluate the chatbots’ accuracy across five oxalate categories: little or none, low, moderate, high, and very high. The performance of each model was analysed, and commonly correct and incorrect classifications were identified. Results: ChatGPT 4.0 demonstrated the highest overall accuracy (69.8%), significantly outperforming Gemini (36.5%) and Microsoft Copilot (26.9%). Foods such as spinach and cocoa were consistently classified correctly, while foods like carrot and walnut were commonly misclassified. Statistical analysis using Cochran’s Q test revealed significant differences in accuracy among the chatbots (p-value <0.05). Conclusion: This study highlights the potential of AI chatbots in dietary management, particularly in supporting clinicians who recommend low oxalate diets for patients with conditions such as hyperoxaluria or kidney disease stones. However, it emphasises the need for further refinement to improve accuracy, especially in classifying foods with regional variations or complex compositions commonly encountered in clinical settings.

Common Sense Reasoning on Fully Localized AI; A Novel systematic Survey

In the modern world of AI, reasoning is the fundamental building block for it, from conversable AI to tasking robots from simple to humanoid, different decisions need to be made on the basis of the various reasoning which includes common sense. The most advanced systems still struggle with higher levels of reasons with common sense for which human is trivial. The paper brings survey on common sense reasoning that can be fully localized to bring in a state of the art frameworks for common sense reasoning to be fully localized. The paper also tries to bring forward that there is more research scope that can be explored. The paper also brings forward how conversational or task oriented and/or both Common Sense motivates research’s to bring in and explore a well-defined framework that can be applied to conversational AI which human like from simple virtual bots to humanoids or any other robots

Convergence of Conversational AI with IoT and AR/VR: A systematic review of emerging applications and industry transformations

Integrating conversational AI with emerging technologies fundamentally transforms human-computer interaction across multiple sectors and applications. This article comprehensively analyzes how conversational AI converges with the Internet of Things (IoT) and Augmented Reality/Virtual Reality (AR/VR) technologies to create more intuitive, responsive, and immersive user experiences. By systematically examining current implementations, we explore the synergistic relationships between these technologies in smart environments, healthcare systems, industrial operations, and educational platforms. The article reveals that while this technological convergence offers unprecedented opportunities for automation, personalization, and enhanced user engagement, it also presents significant challenges regarding system integration, data privacy, and security. The findings indicate that successful implementation requires careful consideration of infrastructure requirements, user experience design, and privacy frameworks. This article contributes to the growing body of knowledge on multi-modal technology integration. It provides insights for researchers, developers, and industry practitioners working towards the next generation of intelligent interactive systems. The article concludes by identifying critical areas for future research and development in this rapidly evolving field.

P-2229. Comparative Evaluation of Multimodal Large Language Models with Vision in Infectious Diseases Cases with Clinical Images

Abstract Background Real-world patient scenarios often involve a composite of text and various clinical images. We explored the performance of advanced multimodal Large Language Models (LLMs) equipped with image-to-text models (vision) in diagnosing complex infectious disease cases containing both text and clinical images.Figure 1:Case #19003; Right foot cutaneous lesion [Photography]. Partners’ Infectious Disease Images; Accessed on April 4, 2024 from https://www.idimages.org/images/detail/?imageid=2356 A sample image from IDImages.org showing a physical exam finding and the descriptions from the AI chatbots. Methods We assessed the performance of two publicly accessible LLMs with vision—GPT-4 and Claude Opus—on a series of 25 complex infectious disease scenarios with 67 clinical images sourced from www.IDImages.org. Prompts were standardized for each case, including instructions to identify the diagnosis, generate differential diagnoses, and describe each image. The models' accuracy in identifying the diagnosis and including it within the differential diagnoses was evaluated using a binary scoring system (0 or 1). Image description quality was assessed using a Likert scale ranging from 0 to 2 comparing the models' outputs against provided descriptions (Figure 1). Scoring was conducted by two independent clinicians, with the final scores being computed as averages. Figure 2 Mean scores of GPT-4 and Opus (1 denoting maximum accuracy) in correctly identifying the diagnosis (left) and in including the main diagnosis in the list of differential diagnoses (right). Results Opus achieved a higher mean accuracy (0.74) compared to GPT-4 (0.56) in determining the correct diagnosis, with the observed difference being statistically significant (p = 0.043). Nevertheless, both models demonstrated comparably high accuracy in incorporating the primary diagnosis within the differential diagnoses, with scores of 0.90 for GPT-4 and 0.88 for Opus (Figure 2). Additionally, both models (GPT-4 average score 1.79 ± 0.43, Opus 1.75 ± 0.50) performed well in providing accurate descriptions of different clinical images. In terms of image type, they performed best in interpreting images with physical findings (GPT-4 1.92 ± 0.23, Opus 1.83 ± 0.46), followed by radiographic images (GPT-4 1.80 ± 0.41, Opus 1.83 ± 0.36), and worst in pathological specimens (GPT-4 1.61 ± 0.58, Opus 1.57 ±0.58) (Figure 3). No statistical differences were found between the two models in their overall ability to describe images or within each image category. Figure 3 Mean accuracy scores of GPT-4 and Opus (up to 2) in interpreting clinical images grouped according to image type. Conclusion The two models demonstrated comparable accuracy in handling complex infectious disease scenarios that integrated both texts and images, suggesting potential utility in enhancing clinical support. Disclosures All Authors: No reported disclosures

Toward cultural interpretability: A linguistic anthropological framework for describing and evaluating large language models

This article proposes a new integration of linguistic anthropology and machine learning (ML) around convergent interests in both the underpinnings of language and making language technologies more socially responsible. While linguistic anthropology focuses on interpreting the cultural basis for human language use, the ML field of interpretability is concerned with uncovering the patterns that Large Language Models (LLMs) learn from human verbal behavior. Through the analysis of a conversation between a human user and an LLM-powered chatbot, we demonstrate the theoretical feasibility of a new, conjoint field of inquiry, cultural interpretability (CI). By focusing attention on the communicative competence involved in the way human users and AI chatbots coproduce meaning in the articulatory interface of human-computer interaction, CI emphasizes how the dynamic relationship between language and culture makes contextually sensitive, open-ended conversation possible. We suggest that, by examining how LLMs internally “represent” relationships between language and culture, CI can: (1) provide insight into long-standing linguistic anthropological questions about the patterning of those relationships; and (2) aid model developers and interface designers in improving value alignment between language models and stylistically diverse speakers and culturally diverse speech communities. Our discussion proposes three critical research axes: relativity, variation, and indexicality.

Consumers’ Intentions to Use AI Chatbots on Online Shopping Platforms

ABSTRACT This study explores the perceptual and psychological factors influencing users’ intentions to use AI chatbots on online shopping platforms. Specifically, it examines the impact of social presence, perceived expertise, and individual differences like introversion on the intentions to use AI chatbots. The study surveyed 213 online shoppers, using SmartPLS 4.0 for path analysis and to test the moderating role of social presence on the relationship between introversion and the intentions to use AI chatbots. The results indicate that introversion, social presence, and perceived expertise are positively associated with the intentions to use AI chatbots. Additionally, perceived expertise is positively associated with social presence, and social presence moderates the relationship between introversion and the intentions to use AI chatbots. These findings offer insights into the psychological factors affecting users’ decisions to engage with AI chatbots, contributing to a deeper understanding of how individual traits and perceptions influence online shopping experiences.

Consumer perceptions of AI chatbots on Twitter (X) and Reddit: an analysis of social media sentiment and interactive marketing strategies

PurposeThis study investigated consumer perceptions of AI chatbots focusing on sentiment analysis across Twitter (X) and Reddit during ChatGPT3 through ChatGPT4 launches. It identifies user sentiments: positive, negative, or neutral and explores their impact on chatbot deployment in interactive marketing. The goal was to understand consumer engagement dynamics and provide insights for enhancing marketing strategies and consumer interactions with chatbots.Design/methodology/approachUsing sentiment analysis, this research examined the nature and scope of discussions surrounding AI chatbots. This methodological approach allowed for a nuanced understanding of the predominant sentiments: positive, negative, or neutral, expressed by users providing insights into consumer engagement and interaction patterns.FindingsThe findings reveal a diverse range of consumer sentiments toward ChatGPT 3, reflecting varying degrees of acceptance and skepticism. These varied sentiments are crucial for organizations in shaping their interactive marketing strategies, particularly in how they deploy chatbots for consumer engagement and brand interaction.Practical implicationsTrust and positive sentiment toward ChatGPT, particularly on platforms like Twitter and Reddit, suggest it is becoming part of everyday life. However, concerns about its impact on human jobs and the lack of emotional intelligence persist. Users still weigh the benefits and drawbacks of ChatGPT, with negative and neutral sentiments reflecting these worries. For interactive marketers, this presents an opportunity to differentiate through human interaction in customer-facing roles. Addressing the risks and ethical concerns of AI, marketers can better engage consumers and refine strategies for future chatbot deployments, ensuring AI enhances rather than detracts from the customer experience.Originality/valueThis paper makes a unique contribution to the existing literature by demonstrating how consumer perceptions, rather than mere acceptance, directly influence the strategic use of AI chatbots in interactive marketing. By focusing on perceptions, this study offers deeper insights from social media sentiment analysis that can refine marketing campaigns and enhance consumer engagement with emerging technologies in the digital landscape.

Proactive Conversational AI: A Comprehensive Survey of Advancements and Opportunities

Dialogue systems are designed to offer human users social support or functional services through natural language interactions. Traditional conversation research has put significant emphasis on a system’s response-ability, including its capacity to understand dialogue context and generate appropriate responses. However, the key element of proactive behavior – a crucial aspect of intelligent conversations – is often overlooked in these studies. Proactivity empowers conversational agents to lead conversations towards achieving pre-defined targets or fulfilling specific goals on the system side. Proactive dialogue systems are equipped with advanced techniques to handle complex tasks, requiring strategic and motivational interactions, thus representing a significant step towards artificial general intelligence. Motivated by the necessity and challenges of building proactive dialogue systems, we provide a comprehensive review of various prominent problems and advanced designs for implementing proactivity into different types of dialogue systems, including open-domain dialogues, task-oriented dialogues, and information-seeking dialogues. We also discuss real-world challenges that require further research attention to meet application needs in the future, such as proactivity in dialogue systems that are based on large language models, proactivity in hybrid dialogues, evaluation protocols and ethical considerations for proactive dialogue systems. By providing a quick access and overall picture of the proactive dialogue systems domain, we aim to inspire new research directions and stimulate further advancements towards achieving the next level of conversational AI capabilities, paving the way for more dynamic and intelligent interactions within various application domains.

Tailoring generative AI chatbots for multiethnic communities in disaster preparedness communication: extending the CASA paradigm

Abstract This study is among the first to develop different prototypes of generative artificial intelligence (GenAI) chatbots powered by GPT-4 to communicate hurricane preparedness information to diverse residents. Drawing from the Computers Are Social Actors paradigm and the literature on disaster vulnerability and cultural tailoring, we conducted a between-subjects experiment with 441 Black, Hispanic, and Caucasian residents of Florida. Our results suggest that GenAI chatbots varying in tone formality and cultural tailoring significantly influence perceptions of their friendliness and credibility, which, in turn, relate to hurricane preparedness outcomes. These results highlight the potential of using GenAI chatbots to improve diverse communities’ disaster preparedness.

AI chatbots in Research: Yes or No? A Self-Reflective Exploration

ChatGPT, an artificial intelligence chatbot released in November 2022, is the fastest-growing consumer application in history. As a generative AI that uses Natural Language Processing, it creates a plethora of content with a ‘human voice’. Unsurprisingly, ChatGPT garnered much attention from academia as it passed several professional exams and has multiple avenues for potential misuse by students and researchers alike. Therefore, this study addresses the dearth in literature by performing a self-reflection study on the practical usage of AI chatbots in research, with a research question: What is the self-reflection of the authors on the usage of AI chatbots for research? This study was framed under the Technology Acceptance Model to provide a comprehensive discussion covering multiple domains. AI chatbots provide advantages and disadvantages to the end-user, but the resulting outcome lies in the hands of the user; hence, educating existing and future users of the tool to use it responsibly should be first and foremost. As Pandora's AI chatbot box has been opened, ethical issues are also plentiful in chatbots. However, it is up to academia to solve these in multidisciplinary settings because, as history has shown, curtailing the use of new technologies is futile. Overall, this study contributes to the body of knowledge in AI Chatbot research by emphasising their potential and addressing probable issues when using them in research.

LLM-based Conversational AI Therapist for Daily Functioning Screening and Psychotherapeutic Intervention via Everyday Smart Devices

Despite the global mental health crisis, access to screenings, professionals, and treatments remains high. In collaboration with licensed psychotherapists, we propose a C onversational AI T herapist with psychotherapeutic I nterventions (CaiTI), a platform that leverages large language models (LLM)s and smart devices to enable better mental health self-care. CaiTI can screen the day-to-day functioning using natural and psychotherapeutic conversations. CaiTI leverages reinforcement learning to provide personalized conversation flow. CaiTI can accurately understand and interpret user responses. When the user needs further attention during the conversation, CaiTI can provide conversational psychotherapeutic interventions, including cognitive behavioral therapy (CBT) and motivational interviewing (MI). Leveraging the datasets prepared by the licensed psychotherapists, we experiment and microbenchmark various LLMs’ performance in tasks along CaiTI's conversation flow and discuss their strengths and weaknesses. With the psychotherapists, we implement CaiTI and conduct 14-day and 24-week studies. The study results, validated by therapists, demonstrate that CaiTI can converse with users naturally, accurately understand and interpret user responses, and provide psychotherapeutic interventions appropriately and effectively. We showcase the potential of CaiTI LLMs to assist the mental therapy diagnosis and treatment and improve day-to-day functioning screening and precautionary psychotherapeutic intervention systems.