Conversational Context Research Articles

Conversational AI: An Explication of Few-Shot Learning Problem in Transformers-Based Chatbot Systems

With the recent advancements in conversational artificial intelligence (AI), the practical applications of chatbots have risen significantly in diverse domains such as healthcare, education, e-government, customer support systems, social platforms, and entertainment. The chatbot converses with humans in natural language and responds to their queries with precise and relevant answers. The relevant literature presents several methods for intent classification and slot mapping for chatbot question answering. However, the existing chatbot architectures still suffer from few-shot learning problem (imbalance ratio of class labels over samples) and ineffective dialog management to retain the context and slots mapping. This study aims to introduce the architecture of chatbot by focusing few-shot learning problem and context management in dialog-based conversations. First, to mitigate the few-shot learning problem with chatbots, we propose a novel hybrid intent and slots transformers (HIST) model. The HIST chatbot architecture utilizes transformers and self-attention mechanisms along with bigated recurrent unit and combines conditional random field algorithms for intent classification and slot extraction. Second, to address dialog management, we introduce a hybrid interaction strategy for slots mapping and effective conversational context management. To validate the proposed model’s effectiveness, comprehensive empirical analysis is carried out using three benchmark datasets including airline travel information system (ATIS), banking77, and conversational language interface for natural conversation 150 (CLINC150). The results show that HIST outperforms against the state-of-the-art existing methods with a clear margin and obtained an accuracy of 94.89% and 96.17% for intent classification and slots extraction, respectively. The empirical results confirm the effectiveness of the HIST chatbot for resolving the few-shot learning problem with effective dialog management in chatbot systems.

Conversational AI

Abstract: Conversational AI systems are becoming increasingly popular across many industries and are transforming the way people interact with technology. For a more authentic, human-like connection and a smooth user experience, these systems should combine text-based interactions with multimodal capabilities. The authors of this work suggest a new approach to improving conversational AI systems' usability by combining speech and visual analysis. By combining visual and auditory processing capabilities, AI systems can better understand human inquiries and instructions. Both visual data and speech can be better understood with the use of computer vision algorithms and natural language processing techniques, respectively. Conversational AI systems can provide more accurate and tailored replies by integrating many modalities to better grasp human intent and context. The development of multimodal conversational AI presents a significant difficulty in ensuring the smooth integration of voice and visual processing units. A strong architectural design and advanced algorithms are necessary for the simultaneous synchronization and comprehension of data from several modalities in real-time. The system needs to keep track of the conversation's context even when it switches between different forms of communication so it can keep providing fair and relevant responses all through the engagement. Customization is key to making multimodal conversational AI better for users. Based on user data and preferences, the system may tailor interactions to offer more relevant ideas and support. Users are more invested in the AI system over time, and they have a better experience overall because to customization. Ensuring the privacy and security of important audiovisual data is of the utmost importance while building multimodal conversational AI. Strong encryption, anonymization technologies, and compliance with data protection regulations are vital for user privacy and system confidence. Continuous improvement is key to the success of multimodal conversational AI systems. The feedback from users can help the developers improve the system and add new features. Thanks to this iterative technique, the AI system stays flexible and can adjust to changing consumer preferences. By combining voice and picture processing, conversational AI systems have a great deal of promise for improving the user experience. Through the integration of visual and auditory signals, these systems have the ability to comprehend user intent more accurately, provide customized experiences, and completely transform the way humans engage with technology.

Leveraging Tokenism or “Pulling Up the Ladder”: Phenomenology of Black Faculty/Student Experiences at HWCUs

ABSTRACT The topic of affirmative action has been a subject of intense debate, with both proponents and opponents offering compelling arguments. Justice Clarence Thomas, an associate justice of the U.S. Supreme Court, has consistently expressed strong reservations about affirmative-action policies, viewing them as a form of reverse discrimination that may “hinder the principles of meritocracy and equal treatment under the law.” However, it is crucial to examine the personalized experiences and consider the nuances, such as underrepresentation and microaggressions, that surround affirmative action. This study utilizes a symbolic interactionist and phenomenological approach that pulls from the individualized/collective experiences of Black students/faculty at historically White colleges and universities (HWCUs) and examines their sociocultural experiences on campus. Findings from this study highlight the continuing discriminative practices and “peer to peer” interactions that Black students/faculty must navigate at HWCUs and the strategies utilized to circumvent mistreatment and institutional adversities. Data from this study provide more context for the ongoing conversation pertaining to affirmative action, “peer to peer,” and institutional discrimination that Black students and faculty experience at HWCUs. In this article, I examine the concept of “leveraging tokenism,” a strategy that Black students and faculty utilize to circumvent underrepresentation and racial microaggressions.

Gemini versus ChatGPT: applications, performance, architecture, capabilities, and implementation

This research paper presents an in-depth comparative examination of Gemini and ChatGPT, two prominent conversational AI models, exploring their respective applications, performance metrics, architectural variances, and overall capabilities. As conversational AI becomes increasingly prevalent across industries, comprehending the nuances of these models becomes pivotal for effective deployment. The paper initiates by outlining the wide array of applications for both Gemini and ChatGPT, spanning industries such as customer service, construction, finance, education, healthcare, and entertainment. It analyzes how each model addresses specific use cases, emphasizing their flexibility and potential impact across different sectors. Following this, the study assesses the performance of Gemini and ChatGPT through both empirical benchmarks and real-world deployment scenarios. Key metrics, including response coherence, accuracy, latency, and scalability, are scrutinized to gauge the models' ability to generate contextually appropriate and coherent responses in conversational contexts. Moreover, the paper elucidates the architectural distinctions between Gemini and ChatGPT, covering variances in training methodologies, model architectures, and underlying technologies. Understanding these architectural nuances provides deeper insights into the computational mechanisms underpinning each model's performance. Lastly, the paper explores the capabilities of Gemini and ChatGPT in handling complex linguistic phenomena, deciphering user intents, and sustaining engaging dialogues over prolonged interactions. This discussion encompasses language generation, sentiment analysis, context retention, and ethical considerations, shedding light on the potential of these models to facilitate meaningful human-computer interactions. Through this thorough comparative analysis, the research contributes to the ongoing conversation surrounding conversational AI systems. It offers valuable insights into the strengths and limitations of Gemini and ChatGPT, empowering stakeholders to make informed decisions regarding their optimal utilization across diverse applications.

Lexical Alignment is Pervasive Across Contexts in Non-WEIRD Adult-Child Interactions.

Lexical alignment, a communication phenomenon where conversational partners adapt their word choices to become more similar, plays an important role in the development of language and social communication skills. While this has been studied extensively in the conversations of preschool-aged children and their parents in Western, Educated, Industrialized, Rich, and Democratic (WEIRD) communities, research in other pediatric populations is sparse. This study makes significant expansions on the existing literature by focusing on alignment in naturalistic conversations of school-aged children from a non-WEIRD population across multiple conversational tasks and with different types of adult partners. Typically developing children aged 5 to 8 years (n = 45) engaged in four semi-structured conversations that differed by task (problem-solving vs. play-based) and by partner (parent vs. university student), resulting in a corpus of 180 conversations. Lexical alignment scores were calculated and compared to sham conversations, representing alignment occurring at the level of chance. Both children and adults coordinated their conversational utterances by re-using or aligning each other's word choices. This alignment behavior persisted across conversational tasks and partners, although the degree of alignment was moderated by the conversational context. These findings suggest that lexical alignment is a robust phenomenon in conversations between school-age children and adults. Furthermore, this study extends lexical alignment findings to a non-WEIRD culture, suggesting that alignment may be a coordination strategy employed by adults and children across diverse linguistic and cultural backgrounds.

An Artificial Intelligence Chatbot is an Accurate and Useful Online Patient Resource Prior to Total Knee Arthroplasty

BackgroundOnline information is a useful resource for patients seeking advice on their orthopaedic care. While traditional websites provide responses to specific frequently asked questions (FAQs), sophisticated artificial intelligence tools may be able to provide the same information to patients in a more accessible manner. Chat Generative Pretrained Transformer (ChatGPT) is a powerful artificial intelligence chatbot that has been shown to effectively draw on its large reserves of information in a conversational context with a user. The purpose of this study was to assess the accuracy and reliability of ChatGPT-generated responses to FAQs regarding total knee arthroplasty. MethodsWe distributed a survey that challenged arthroplasty surgeons to identify which of the 2 responses to FAQs on our institution’s website was human-written and which was generated by ChatGPT. All questions were total knee arthroplasty–related. The second portion of the survey investigated the potential to further leverage ChatGPT to assist with translation and accessibility as a means to better meet the needs of our diverse patient population. ResultsSurgeons correctly identified the ChatGPT-generated responses 4 out of 10 times on average (range: 0 to 7). No consensus was reached on any of the responses to the FAQs. Additionally, over 90% of our surgeons strongly encouraged the use of ChatGPT to more effectively accommodate the diverse patient populations that seek information from our hospital’s online resources. ConclusionsChatGPT provided accurate, reliable answers to our website’s FAQs. Surgeons also agreed that ChatGPT’s ability to provide targeted, language-specific responses to FAQs would be of benefit to our diverse patient population.

Hypergraph Neural Network for Emotion Recognition in Conversations

Modeling conversational context is an essential step for emotion recognition in conversations. Existing works still suffer from insufficient utilization of local context information and remote context information. This article designs a hypergraph neural network, namely HNN-ERC, to better utilize local and remote contextual information. HNN-ERC combines the recurrent neural network with the conventional hypergraph neural network to strengthen connections between utterances and make each utterance receive information from other utterances better. The proposed model has empirically achieved state-of-the-art results on three benchmark datasets, demonstrating the effectiveness and superiority of the new model.

The semantics of gaze in person perception: a novel qualitative-quantitative approach

Interpreting gaze behavior is essential in evaluating interaction partners, yet the ‘semantics of gaze’ in dynamic interactions are still poorly understood. We aimed to comprehensively investigate effects of gaze behavior patterns in different conversation contexts, using a two-step, qualitative-quantitative procedure. Participants watched video clips of single persons listening to autobiographic narrations by another (invisible) person. The listener’s gaze behavior was manipulated in terms of gaze direction, frequency and direction of gaze shifts, and blink frequency; emotional context was manipulated through the valence of the narration (neutral/negative). In Experiment 1 (qualitative-exploratory), participants freely described which states and traits they attributed to the listener in each condition, allowing us to identify relevant aspects of person perception and to construct distinct rating scales that were implemented in Experiment 2 (quantitative-confirmatory). Results revealed systematic and differential meanings ascribed to the listener’s gaze behavior. For example, rapid blinking and fast gaze shifts were rated more negatively (e.g., restless and unnatural) than slower gaze behavior; downward gaze was evaluated more favorably (e.g., empathetic) than other gaze aversion types, especially in the emotionally negative context. Overall, our study contributes to a more systematic understanding of flexible gaze semantics in social interaction.

Implicature priming, salience, and context adaptation

Recent experimental research has observed two kinds of priming effects on quantity implicatures. One is the Strong–Weak contrast, where more quantity implicatures are observed after prime trials forcing interpretations with quantity implicatures (‘Strong primes’) than after prime trials forcing interpretations without quantity implicatures (‘Weak primes’). The other effect is the Alternative-Weak contrast, where prime trials mentioning alternative expressions (‘Alternative primes’) similarly lead to more quantity implicatures. It has been claimed that both of these effects should be understood in terms of increased salience of alternative expressions used to compute quantity implicatures. We present experimental evidence that speaks against this hypothesis. With the help of novel baseline conditions, which were absent in previous studies on implicature priming, we observe that the results in the priming paradigm commonly used in the literature are inverse preference effects in the sense that robust priming effects are observed towards interpretations that are normally unexpected, and depending on the baseline expectation, each of the three prime types mentioned above may have priming effects. We furthermore investigated different types of alternative priming for so-called ad hoc implicatures and found that for these implicatures, presenting an alternative expression in a simple sentence does not have a priming effect on the implicature of a similarly simple sentence, but presenting it in a more complex conjunctive construction does. Our results also show that conjunctions of similar but irrelevant expressions have a similarly robust priming effect and that conjunctive sentences with two conjuncts do not give rise to priming effects on the interpretation of sentences of the same syntactic complexity, but those with three conjuncts do. To make sense of these observations, we propose that what crucially matters for priming implicatures is incremental change in one’s probabilistic expectations about the current conversational context brought about by a process we call context adaptation.

"BORN INTO THE WRONG FAMILY": NAVIGATING BEING A BLACK MALE IN STEM

The underrepresentation of African Americans in science, technology, engineering, and mathematics (STEM) degree programs and professional spaces has been a national concern for years. This phenomenon has prompted the development and implementation of various programs and initiatives to increase access to and sustain their participation in STEM. Of considerable importance, African American men only represent three percent of the scientists and engineers working in the field. However, there are few studies that have explored the academic and professional experiences of African American men through critical, holistic perspectives. This project utilizes critical race theory (CRT) and phenomenological variant ecological systems theory (PVEST) to magnify the experiences of Black men in STEM across academic and professional ranks. Through this combined CRT and PVEST lens, analyzing the challenges and navigational strategies of Black men in STEM, we reveal that Black men in STEM minimize their vulnerability to stereotypes and biases through resistance and resilience. Moreover, the utilization of a virtual counterspace as the research context for conversation between the participants provided a unique interventional approach that fostered rich intergenerational dialogue that enhanced both navigational and resistance capital for the participants. Implications for research and praxis are provided.

Contextualism as a Solution to the Gettier Problem: Revisiting the Definition and Justification of Knowledge

The Gettier problem has significantly reshaped the discourse on the nature of knowledge, challenging the traditional definition of knowledge as Justified True Belief (JTB). This paper explores the philosophical implications of the Gettier problem and its impact on epistemology, focusing on two major responses: internalism and contextualism. Internalism emphasizes the cognitive subject's internal states as the basis for knowledge justification, while contextualism introduces variability in knowledge standards based on conversational and situational contexts. The strengths and limitations of each approach are critically analyzed, with internalism offering clear criteria for justification but struggling with external factors, and contextualism providing flexibility in knowledge attribution while risking subjectivity. The paper also examines how contextualism addresses philosophical skepticism and the Gettier problem by adjusting the criteria for knowledge depending on the context. Ultimately, this manuscript argues that while both internalism and contextualism have contributed to a deeper understanding of knowledge, further refinement of these theories is necessary to address their limitations, particularly regarding objectivity and skepticism.

Knowledge Graphs and Pretrained Language Models Enhanced Representation Learning for Conversational Recommender Systems.

Conversational recommender systems (CRSs) utilize natural language interactions and dialog history to infer user preferences and provide accurate recommendations. Due to the limited conversation context and background knowledge, existing CRSs rely on external sources such as knowledge graphs (KGs) to enrich the context and model entities based on their interrelations. However, these methods ignore the rich intrinsic information within entities. To address this, we introduce the knowledge-enhanced entity representation learning (KERL) framework, which leverages both the KG and a pretrained language model (PLM) to improve the semantic understanding of entities for CRS. In our KERL framework, entity textual descriptions are encoded via a PLM, while a KG helps reinforce the representation of these entities. We also employ positional encoding to effectively capture the temporal information of entities in a conversation. The enhanced entity representation is then used to develop a recommender component that fuses both entity and contextual representations for more informed recommendations, as well as a dialog component that generates informative entity-related information in the response text. A high-quality KG with aligned entity descriptions is constructed to facilitate this study, namely, the Wiki Movie Knowledge Graph (WikiMKG). The experimental results show that KERL achieves state-of-the-art results in both recommendation and response generation tasks. Our code is publicly available at the link: https://github.com/icedpanda/KERL.

Komunikacijske sposobnosti dece hrvatske i romske nacionalnosti s lakim intelektualnim teškoćama - postignuće na Ček-listi dečje komunikacije (CCC)

Introduction. Communication and language development in children with (mild) intellectual disability is generally delayed. Roma national minority children are sequential bilinguals, most introduced to the Croatian language upon entering the educational system. Information on communication in the natural context can be obtained through checklists completed by children's communication partners. Objectives. This study aims to obtain insight into the communication and Croatian language abilities of Croatian and Roma children with mild intellectual disabilities. Method. The study participants were 52 children between 9 and 16 years old (22 Croatian and 30 Roma). Their performance on the Children's Communication Checklist was analyzed and compared to published thresholds and each other. Results. The performance of Croatian and Roma children on the pragmatic composite is comparable to that of British peers with intellectual disability, as both groups scored below the normal range. Only the Roma children performed below clinical thresholds on the Speech and Syntax scale. No group displayed autism features. Mann-Whitney test showed significant differences between the groups in Speech output and Syntax subscales, indicating Roma children's poorer Croatian language abilities. Overall pragmatic abilities did not differ between the groups. Both groups scored below the threshold on the Coherence and Use of conversational context subscales, showing comparable pragmatic profiles. However, Croatian children outperformed Roma children on the Coherence, Use of conversational context, and Conversational rapport subscales. Conclusion. Roma children use the Croatian language in a way comparable to their Croatian peers despite being less proficient in its structure. Pragmatic abilities should be targeted in children with intellectual disabilities.

Enhancing English For Nursing: A Comprehensive Needs Analysis For Effective Communication In Emergency Nursing Study Programs

students enrolled in nursing study programs with a focus on emergency characteristics. The investigation encompasses both target needs and learning needs, emphasizing the specific language requirements in the nursing field. English for Nursing falls under the domain of English for Specific Purposes (ESP), designed to cater to learners' specific language needs, fostering more effective and comprehensive teaching and learning. By aligning English instruction with the demands of the nursing discipline, non-English program students will gain familiarity with English tailored to nursing contexts. The ESP approach aims to address the specific language acquisition challenges faced by learners with distinct objectives. Data collection involves questionnaires administered to 45 students and direct interviews with emergency nursing lecturers. Findings reveal the significance of learning English for improved communication within the context of conversations or practical applications. The identified English materials concentrate on terms relevant to nursing study programs, particularly those associated with emergency characteristics, becoming the focal points of instruction. Additionally, materials focusing on enhancing listening and speaking skills emerge as key priorities in the learning process. The outcomes of this study are anticipated to serve as a valuable reference for the design and development of syllabi, specifically tailored to English for Nursing courses.

Examining the Impact of Digital Human Gaze Expressions on Engagement Induction.

With advancements in technology, digital humans are becoming increasingly sophisticated, with their application scope widening to include interactions with real people. However, research on expressions that facilitate natural engagement in interactions between real people and digital humans is scarce. With this study, we aimed to examine the differences in user engagement as measured by subjective evaluations, eye tracking, and electroencephalogram (EEG) responses relative to different gaze expressions in various conversational contexts. Conversational situations were categorized as face-to-face, face-to-video, and digital human interactions, with gaze expressions segmented into eye contact and gaze avoidance. Story stimuli incorporating twelve sentences verified to elicit positive and negative emotional responses were employed in the experiments after validation. A total of 45 participants (31 females and 14 males) underwent stimulation through positive and negative stories while exhibiting eye contact or gaze avoidance under each of the three conversational conditions. Engagement was assessed using subjective evaluation metrics in conjunction with measures of the subjects' gaze and brainwave activity. The findings revealed engagement disparities between the face-to-face and digital-human conversation conditions. Notably, only positive stimuli elicited variations in engagement based on gaze expression across different conversation conditions. Gaze analysis corroborated the engagement differences, aligning with prior research on social sensitivity, but only in response to positive stimuli. This research departs from traditional studies of un-natural interactions with digital humans, focusing instead on interactions with digital humans designed to mimic the appearance of real humans. This study demonstrates the potential for gaze expression to induce engagement, regardless of the human or digital nature of the conversational dyads.

Companion animal veterinarians discuss aspects of one health with pet owners during most veterinary appointments.

To examine the prevalence and context of one-health conversations between veterinarians and clients in companion animal practice. A random selection of 60 companion animal veterinarians; a convenience sample of 917 interactions from Southern Ontario, Canada. Of these, 100 audio-video-recorded interactions including 47 of 60 veterinarians were randomly selected for inclusion in this study. Audio-video recordings were made of veterinarian-client-patient interactions between November 2017 and January 2019. A researcher-generated coding framework was developed and used to assess the prevalence and content of one-health topics communicated during veterinary appointments. Of the interactions assessed, 60 were preventive care and 40 were health problem appointments. Further, 78% (78/100) included at least 1 discussion related to one health. One-health topics included zoonoses (28% [28/100]), animal behavior (25% [25/100]), illness/disease (20% [20/100]), activity level/exercise (16% [16/100]), nutrition (16% [16/100]), dentistry (6% [6/100]), body weight (3% [3/100]), animal welfare (3% [3/100]), dog/cat bites (2% [2/100]), cannabis (2% [2/100]), and aging (1% [1/100]). Zoonotic diseases were mentioned in 65 appointments, 28 of which evolved into a one-health discussion. Antibiotics were discussed in 27 appointments, none of which were discussed in relation to one health (eg, antimicrobial resistance). Findings suggest that one-health topics are raised within most veterinary appointments. Opportunities exist for more comprehensive one-health conversations between veterinarians and their clients, particularly in relation to zoonotic diseases and antimicrobials.

Sarcasm Detection in News Headline Dataset with Ensemble Deep Learning Method

Sarcasm, a prevalent linguistic device, is frequently used in public discourse, often causing offence and distress to the listener. The complexity inherent in detecting sarcasm is a significant and ongoing challenge in the field of sentiment analysis research. The widespread use of this phenomenon in diverse conversational contexts further complicates its identification in data sets full of human interactions. Deficiencies in methodologies for distinguishing such statements adversely affect the performance of sentiment analysis, especially in distinguishing negative, positive or neutral sentiments. Inaccuracies in sarcasm detection can affect the classification results of sentiment analysis. Therefore, sentiment analysis seeks to categorise sarcastic sentences that, despite appearing positive, actually contain negative meanings. This research aims to build a deep learning ensemble stack model. The basic deep learning methods used are Bidirectional Gated Recurrent Unit (BiGRU) and Convolutional Neural Network (CNN). LightGBM is used to perform stack ensemble of deep learning methods. The dataset used comes from the Kaggle website and consists of English headlines. The findings show that the stack ensemble method outperforms BiGRU and CNN, evidenced by an accuracy rate of 91.2% and an F1 score of 90.2%. Therefore, from the above discussion, it can be concluded that the LightGBM method emerges as the optimal solution for sarcasm detection

은행권의 경쟁력 확보를 위한 ChatGPT 활용방안 연구

Recently, the banking industry has been implementing AI bankers utilizing artificial intelligence on-site. However, the application of AI bankers has revealed various technical issues, prompting efforts by banks to address and improve them. In this paper, we propose the combination of AI bankers and ChatGPT as one method of improvement. It is anticipated that this combination will alleviate some of the identified issues and bring about diverse changes in banking services. Specifically, we suggest enabling AI bankers to handle customer interactions by leveraging ChatGPT's ability to store and recall previous conversation context. Additionally, we propose expanding the scope of AI bankers by combining them with ChatGPT trained on regional dialects to recognize and incorporate regional expressions. This approach is expected to enhance productivity and efficiency by bankers focusing on other core tasks. Moreover, the introduction of ChatGPT for employees is expected to improve the quality of banking services through accurate task processing, ultimately increasing customer satisfaction.

Center-retained fine-tuning for conversational question ranking through unsupervised center identification

Given a conversation context, conversational question ranking (CQR) aims to select a proper question from a candidate pool to clarify users’ ambiguous information needs. Most state-of-the-art CQR methods are pre-training based, which seek to design various fine-tuning tasks by randomly manipulating utterances in the context. However, existing fine-tuning methods will occasionally remove the center utterances (i.e., the referent utterances in the context for the clarification questions), which might result in counterproductive effect as the center utterances have high semantic coherence with the clarification questions. In this work, we introduce an unsupervised center-aware ranking (UCAR) framework, which first identifies the center utterance and then conducts fine-tuning by retaining the center. To identify the center, we devise a multi-perspective center identification (MCD) module and optimize it in an unsupervised manner. Experimental results on two benchmark datasets with 20k test samples show that seven state-of-the-art baselines get improved by 1% ∼5% in terms of MRR equipped with UCAR. The results demonstrate that it indeed improves the performance by keeping semantic coherence in conversations when fine-tuning. In addition, the centers identified by our proposed unsupervised MCD module are of high quality, as validated by humans. Moreover, an ancillary advantage of UCAR is that it has good interpretability by referring to the center utterance in the conversation context.

Language Artificial Intelligences' Communicative Performance Quantified Through the Gricean Conversation Theory.

This study pragmatically investigates an artificial intelligence (AI) speaker (AIS)'s verbal communicative performance based on real AI-human conversation data. Specifically, this study explores Grice's conversation theory, which enables the categorization of an AIS's mistaken utterances as violations of specific conversational maxims. Twenty native Korean-speaking participants recorded at least 50 conversations with Kakao Mini AISs, provided by Daum Kakao, Inc., in Korea. Each conversation, either for information sharing or as daily dialogue, was required to contain at least two turn-taking instances. A total of 1,026 recorded dialogues were decomposed into adjacency pairs based on turn-taking. The dialogues were arranged into 3,365 adjacency pairs, and each pair was then classified as a conversational success or failure based on whether the AIS answered the user's utterance appropriately. Language users' evaluations of the AIS's mistaken expressions were also quantified via an additional acceptability rating test with 1,024 adjacency pairs. The overall results indicate that Grice's "maxim of relation" is most frequently flouted by AISs and is considered to be the least natural to language users. These findings suggest that to improve AISs' natural communication capacity, more detailed AI algorithms that generate utterances relevant to either the partner's preceding utterance or a broader conversational context should be created. Although the verbal communicative capacities of the AIS we test are substantially overtaken by those of recent large language models, such as generative pretrained transformer, the pragmatic evaluation described in the current study will remain useful for more precise linguistic quantification of current/future language AI's communicative performance/competence.