Conversational Language Research Articles

Let’s Talk Games: An Expert Exploration of Speech Interaction with NPCs

Recent years have witnessed significant advances in speech recognition and language processing technologies, enabling natural language conversations with computers. Concurrently, the gaming industry seeks to heighten immersion as one of the leading mediums for entertainment. This work investigates the potential and challenges of using speech interaction in single-player video games, particularly for interactions with NPCs. We conducted an online survey with video game experts ( N = 20 ) alongside in-depth interviews with researchers specializing in conversational user interfaces and game user research ( N = 16 ). Our findings emphasize experts’ recognition of the considerable potential of speech interaction in games, fostering increased immersion, engagement, and entertainment. Additionally, experts address pertinent concerns like privacy issues and play environment limitations. Drawing from our findings, we provide practical recommendations for integrating speech interaction in single-player games. These encompass potential benefits, challenges, accessibility, and social implications. We further address potential regulatory requirements and offer implementation tips to enhance player experience.

Mental state verb use in play by preschool-age children who stutter and their mothers

IntroductionPreschool-age children use mental state verbs (MSVs; e.g., think, know) to reference thoughts and other cognitive states. In play-based language, MSV use requires conversational flexibility, as speakers shift from discussion of actions happening in the here-and-now to more abstract discussion of mental states. Some evidence suggests that children who stutter (CWS) demonstrate subtle differences in shifting on experimental tasks of cognitive flexibility, differences which may extend to conversational flexibility. This study explored MSV use in conversational language between CWS and their mothers. MethodsThirty-five preschool-age CWS and 35 age- and gender-matched children who do not stutter (CWNS), all performing within the typical range on standardized language testing, conversed with their mothers during play. Samples were transcribed and coded for MSV use. ResultsNo between-group differences were observed in MSV use, either between the CWS and CWNS or between the groups of mothers. Age and language skills were positively associated with MSV use in the CWNS group only. For both groups of dyads, mothers’ MSV use corresponded at least to some extent to their children’s language skills. Finally, correspondence between CWNS and their mothers was observed for two conversational language measures, representing lexical diversity and morphosyntax; this overall pattern was not observed in the CWS dyad group. ConclusionsAlthough these findings point to similar use of MSVs among the groups of children and their mothers, for the CWS group, the patterns of use in relation to age and language skills are somewhat different from developmental expectations.

ChatGPT/GPT-4 (large language models): Opportunities and challenges of perspective in bariatric healthcare professionals.

ChatGPT/GPT-4 is a conversational large language model (LLM) based on artificial intelligence (AI). The potential application of LLM as a virtual assistant for bariatric healthcare professionals in education and practice may be promising if relevant and valid issues are actively examined and addressed. In general medical terms, it is possible that AI models like ChatGPT/GPT-4 will be deeply integrated into medical scenarios, improving medical efficiency and quality, and allowing doctors more time to communicate with patients and implement personalized health management. Chatbots based on AI have great potential in bariatric healthcare and may play an important role in predicting and intervening in weight loss and obesity-related complications. However, given its potential limitations, we should carefully consider the medical, legal, ethical, data security, privacy, and liability issues arising from medical errors caused by ChatGPT/GPT-4. This concern also extends to ChatGPT/GPT -4's ability to justify wrong decisions, and there is an urgent need for appropriate guidelines and regulations to ensure the safe and responsible use of ChatGPT/GPT-4.

Repair in Ghanaian judicial discourse

Abstract Repair is an integral part of every natural human discursive practice due to cognitive anxiety, stress, emotional valence, inability to recover words from one’s mental lexicon, slip of the tongue, among others. This study examined repair in Ghanaian judicial interactions using authentic judicial discourse data and working within the theoretical frameworks of conversational analysis and language and power. Results indicate that repairables include word search, correction proper and misstatements. Repair types included self-initiated self-repair, other-initiated other-repair, other-initiated self-repair, and other-initiated other-repair and self-repair. Linguistic and discourse-pragmatic strategies employed in initiating and carrying out repair include, pauses, sound prolongation, word/expression repetition, and morpho-syntactic and discourse-pragmatic features like yes-no questions, wh-questions, quantifiers, adjectives denoting exactness, pronouns, laughter, supportives, scolding via pejorative utterances, politeness/address forms, and calling on actors to abide by the courts’ moral code. In sum, repair is important on issues relating to understandability and interpretability of what judicial participants say, acceptable modes communication and their impact on facts needed to make cases judicable. Stakeholders in Ghanaian judicial practice and Ghanaian jurisprudence must take note of repair’s un-expendable nature in Ghanaian judicial discourse.

Chatbots Utility in Healthcare Industry: An Umbrella Review

Introduction: Chatbots, computer programs emulating natural language conversations, have gained attention in healthcare. Recent advances address issues like obesity, dementia, oncology, and insomnia. A comprehensive assessment of their utility is essential for widespread adoption. This study aims to summarize chatbots' role in healthcare.Material and Methods: The methodology involved a systematic review of English-language literature up to May 8, 2023, from databases of Embase, PubMed, Web of Science, and Scopus. Selection followed a two-step process based on inclusion/exclusion criteria. The PRISMA checklist and AMSTAR-2 tool ensured quality.Results: The review encompassed 38 articles. Findings reveal chatbots primarily promote healthy lifestyles, improving mental well-being. They are widely used for treatment, education, and screening due to their accessibility.Conclusion: Chatbots hold transformative potential in healthcare, especially in mental health, cancer management, and public health. They are poised to revolutionize the industry, offering innovative solutions and improving patient outcomes.

SVCGAN: Speaker Voice Conversion Generative Adversarial Network for Children’s Speech Conversion and Recognition

Automatic speech recognition (ASR) refers to a technological process that entails the conversion of spoken language into written text. However, the acoustic distinctions between children’s speech and adult speech are substantial, rendering the automatic speech recognition system trained on adult speech inadequate for effectively recognizing children’s speech. To overcome this issue, in this study, we propose speaker conversion generative adversarial network (SVCGAN). SVCGAN is a novel non-parallel voice conversion model, which enhances three key areas: log-cosh loss, semantic-similarity loss, and third adversarial loss. Therefore, the incorporation of these losses better protects semantic information for young children during voice conversion process and improves the quality of the converted speech. Additionally, the character error rate (CER) of children’s speech recognition can benefit from children’s speech transformed into adult speech. Experimental results suggest that SVCGAN demonstrates superior performance across multiple dimensions compared to both CycleGAN-VC3 and MaskCycleGAN-VC models. It encompasses training efficiency, semantic information similarity, voice type similarity, sound naturalness and intelligibility, which leads to a reduction in the CER of speech recognition for young children.

Enhancing Lip Reading: A Deep Learning Approach with CNN and RNN Integration

This research introduces an innovative approach to enhance lip reading-based text extraction and translation through the integration of a double Convolutional Neural Network (CNN) coupled with Recurrent Neural Network (RNN) architecture. The proposed model aims to leverage the strengths of both CNN and RNN to achieve superior accuracy in lip movement interpretation and subsequent text extraction. The methodology involves training the double CNN+RNN model on extensive datasets containing synchronized lip movements and corresponding linguistic expressions. The initial layers of the model utilize CNNs to effectively capture spatial features from the visual input of lip images. The extracted features are then fed into RNN layers, allowing the model to grasp temporal dependencies and contextual information crucial for accurate lip reading. The trained model showcases its proficiency in extracting textual content from spoken words, demonstrating an advanced capability to decipher nuances in lip gestures. Furthermore, the extracted text undergoes a translation process, enabling the conversion of spoken language into various target languages. This research not only contributes to the advancement of lip reading technologies but also establishes a robust foundation for real-world applications such as accessibility solutions for individuals with hearing impairments, real-time multilingual translation services, and improved communication in challenging acoustic environments. The abstract concludes with a discussion on the potential impact of the double CNN+RNN model in pushing the boundaries of human-computer interaction, emphasizing the synergy between deep learning, lip reading, and translation technologies

Conversational AI: An Explication of Few-Shot Learning Problem in Transformers-Based Chatbot Systems

With the recent advancements in conversational artificial intelligence (AI), the practical applications of chatbots have risen significantly in diverse domains such as healthcare, education, e-government, customer support systems, social platforms, and entertainment. The chatbot converses with humans in natural language and responds to their queries with precise and relevant answers. The relevant literature presents several methods for intent classification and slot mapping for chatbot question answering. However, the existing chatbot architectures still suffer from few-shot learning problem (imbalance ratio of class labels over samples) and ineffective dialog management to retain the context and slots mapping. This study aims to introduce the architecture of chatbot by focusing few-shot learning problem and context management in dialog-based conversations. First, to mitigate the few-shot learning problem with chatbots, we propose a novel hybrid intent and slots transformers (HIST) model. The HIST chatbot architecture utilizes transformers and self-attention mechanisms along with bigated recurrent unit and combines conditional random field algorithms for intent classification and slot extraction. Second, to address dialog management, we introduce a hybrid interaction strategy for slots mapping and effective conversational context management. To validate the proposed model’s effectiveness, comprehensive empirical analysis is carried out using three benchmark datasets including airline travel information system (ATIS), banking77, and conversational language interface for natural conversation 150 (CLINC150). The results show that HIST outperforms against the state-of-the-art existing methods with a clear margin and obtained an accuracy of 94.89% and 96.17% for intent classification and slots extraction, respectively. The empirical results confirm the effectiveness of the HIST chatbot for resolving the few-shot learning problem with effective dialog management in chatbot systems.

CAVEN: An Embodied Conversational Agent for Efficient Audio-Visual Navigation in Noisy Environments

Audio-visual navigation of an agent towards locating an audio goal is a challenging task especially when the audio is sporadic or the environment is noisy. In this paper, we present CAVEN, a Conversation-based Audio-Visual Embodied Navigation framework in which the agent may interact with a human/oracle for solving the task of navigating to an audio goal. Specifically, CAVEN is modeled as a budget-aware partially observable semi-Markov decision process that implicitly learns the uncertainty in the audio-based navigation policy to decide when and how the agent may interact with the oracle. Our CAVEN agent can engage in fully-bidirectional natural language conversations by producing relevant questions and interpret free-form, potentially noisy responses from the oracle based on the audio-visual context. To enable such a capability, CAVEN is equipped with: i) a trajectory forecasting network that is grounded in audio-visual cues to produce a potential trajectory to the estimated goal, and (ii) a natural language based question generation and reasoning network to pose an interactive question to the oracle or interpret the oracle's response to produce navigation instructions. To train the interactive modules, we present a large scale dataset: AVN-Instruct, based on the Landmark-RxR dataset. To substantiate the usefulness of conversations, we present experiments on the benchmark audio-goal task using the SoundSpaces simulator under various noisy settings. Our results reveal that our fully-conversational approach leads to nearly an order-of-magnitude improvement in success rate, especially in localizing new sound sources and against methods that use only uni-directional interaction.

FacetCRS: Multi-Faceted Preference Learning for Pricking Filter Bubbles in Conversational Recommender System

The filter bubble is a notorious issue in Recommender Systems (RSs), which describes the phenomenon whereby users are exposed to a limited and narrow range of information or content that reinforces their existing dominant preferences and beliefs. This results in a lack of exposure to diverse and varied content. Many existing works have predominantly examined filter bubbles in static or relatively-static recommendation settings. However, filter bubbles will be continuously intensified over time due to the feedback loop between the user and the system in the real-world online recommendation. To address these issues, we propose a novel paradigm, Multi-Facet Preference Learning for Pricking Filter Bubbles in Conversational Recommender System (FacetCRS), which aims to burst filter bubbles in the conversational recommender system (CRS) through timely user-item interactions via natural language conversations. By considering diverse user preferences and intentions, FacetCRS automatically model user preference into multi-facets, including entity-, word-, context-, and review-facet, to capture diverse and dynamic user preferences to prick filter bubbles in the CRS. It is an end-to-end CRS framework to adaptively learn representations of various levels of preference facet and diverse types of external knowledge. Extensive experiments on two publicly available benchmark datasets demonstrate that our proposed method achieves state-of-the-art performance in mitigating filter bubbles and enhancing recommendation quality in CRS.

Tackling Vision Language Tasks through Learning Inner Monologues

Visual language tasks such as Visual Question Answering (VQA) or Visual Entailment (VE) require AI models to comprehend and reason with both visual and textual content. Driven by the power of Large Language Models (LLMs), two prominent methods have emerged: (1) the hybrid integration between LLMs and Vision-Language Models (VLMs), where visual inputs are firstly converted into language descriptions by VLMs, serving as inputs for LLMs to generate final answer(s); (2) visual feature alignment in language space, where visual inputs are encoded as embeddings and projected to LLMs' language space via further supervised fine-tuning. The first approach provides light training costs and interpretability but is hard to be optimized in an end-to-end fashion. The second approach presents decent performance, but feature alignment usually requires large amounts of training data and lacks interpretability. To tackle this dilemma, we propose a novel approach, Inner Monologue Multi-Modal Optimization (IMMO), to solve complex vision language problems by simulating Inner Monologue, a cognitive process in which an individual engages in silent verbal communication with themselves. More specifically, we enable LLMs and VLMs to interact through natural language conversation (i.e., Inner Monologue) and propose to use a two-stage training process to learn how to do Inner Monologue (self-asking questions and answering questions). IMMO is evaluated on two popular tasks and achieves competitive performance with less training data when compared with state-of-the-art models while concurrently keeping the interpretability. The results suggest that by emulating the cognitive phenomenon of internal dialogue, our approach can enhance reasoning and explanation abilities, contributing to the more effective fusion of vision and language models. More importantly, instead of using predefined human-crafted monologues, IMMO learns this process within the deep learning models, broadening its potential applications across various AI challenges beyond vision and language tasks.

Indian Sign Language Recognition using transfer learning with Efficient Net

ABSTRACT Automatic conversion of sign language into text or speech can be really useful in communicating among deaf or mute people with no sign language knowledge. The present need is to create an automated system that can translate ISL signals into regular text and vice versa. It is advantageous for both groups to communicate their needs to one another while using publicly accessible services, such as banking, traffic signaling, and ticketing. In this manuscript, Indian Sign Language Recognition using Transfer Learning with Efficient Net (ISLR-EfficientNetB0) is proposed for Indian sign language recognition. Initially, the input images are gathered via the Indian Sign Language (ISL) dataset. The input image is pre-processing by Fairness-aware Collaborative Filtering (FACF) method. In pre-processing stage noise are removed from input images and image resizing is done. Then, Efficient Net B0 is employed to identify ISL. The proposed model is implemented on Python. Then the efficiency of the proposed approach is compared with existing models. The accuracy, precision, specificity, sensitivity, F1-score, computational time, error rate and RoC of the suggested approaches are all assessed. The proposed ISLR-EfficientNetB0 method attains 31.42%, 26.04%, and 20.45% higher accuracy, 19.24%, 26.35%, and 24.02% higher precision, 15.84%, 18.91%, and 28.16% higher specificity and 17.06%, 21.36%, and 32.47% higher sensitivity compared with existing methods, such as Indian sign language recognition under machine learning methods (ISLR-KNN), ISL recognition scheme utilizing SURF with SVM and CNN (ISLR-SVM-CNN) and Benchmarking deep neural network methods for ISL recognition (ISLR-VGG16) respectively.

Language matters for biodiversity

Abstract Language is central to the way people learn about the natural world. A salient concern of the biodiversity conservation arena has been to understand how language can be employed by scientists to communicate knowledge to nonexpert audiences and build ecological literacy. The use of analogy and narrative by scientists are prominent techniques. In this article, we consider how these two modes of language-based reasoning extend into ordinary conversational language use by the public, specifically when articulating everyday understanding and experiences of biodiversity. Drawing on a process of public engagement in a UK woodland environment, a typological framework based on principles of analogical and narrative reasoning is developed to characterize the precise character of processes of everyday biodiversity sense making. The implications of the framework are discussed in the context of future biodiversity research, particularly its participatory and educational dimensions.

The Potential Applications and Challenges of ChatGPT in the Medical Field.

ChatGPT, an AI-driven conversational large language model (LLM), has garnered significant scholarly attention since its inception, owing to its manifold applications in the realm of medical science. This study primarily examines the merits, limitations, anticipated developments, and practical applications of ChatGPT in clinical practice, healthcare, medical education, and medical research. It underscores the necessity for further research and development to enhance its performance and deployment. Moreover, future research avenues encompass ongoing enhancements and standardization of ChatGPT, mitigating its limitations, and exploring its integration and applicability in translational and personalized medicine. Reflecting the narrative nature of this review, a focused literature search was performed to identify relevant publications on ChatGPT's use in medicine. This process was aimed at gathering a broad spectrum of insights to provide a comprehensive overview of the current state and future prospects of ChatGPT in the medical domain. The objective is to aid healthcare professionals in understanding the groundbreaking advancements associated with the latest artificial intelligence tools, while also acknowledging the opportunities and challenges presented by ChatGPT.

Growth of Complex Syntax: Coordinate and Subordinate Clause Use in Elementary School-Aged Children.

Production of complex syntax is a hallmark of later language development; however, most of the research examining age-related changes has focused on adolescents or analyzed narrative language samples. Research documenting age-related changes in the production of complex syntax in elementary school-aged children in conversational language samples is limited. Therefore, the purpose of this article is to examine age-related changes in the production of coordinate and subordinate clauses in children between 5 and 10 years of age obtained from 50-utterance conversational language samples. The analytic sample included 196 children with typical language development, who ranged in age from 5;0 to 10;11 (years;months; girls = 103; boys = 96; three cases were excluded). Fifty-utterance conversational language samples were examined for use of coordinate and subordinate clauses. Results of regression analyses indicated that the production of coordinate and subordinate clauses could be predicted from age. The proportion of utterances that included subordinate clauses increased 0.20% for every month increase in age (p < .001). Coordinate clauses also continued to grow, although at a slower rate (0.10% increase for every month increase in age, p < .001). Finally, the proportion of simple utterances (i.e., utterances without coordinate or subordinate clauses) decreased with age (0.40% decrease for every month increase in age, p < .001). This study indicated that as children's age increased, they used fewer, simple, one-clause sentences and more utterances that included subordinate clauses, with or without coordinate clauses. These results were obtained from 50-utterance language samples, further supporting use of language sampling to develop intervention goals and monitor progress in therapy. https://doi.org/10.23641/asha.25262725.

MULTILINGUAL CODE-SWITCHING IN THE CLASSROOM: AN ANALYSIS FROM THE TEACHER’S NARRATIVE

Code switching, the practice of alternating between languages in conversations, was of interest to researchers in sociolinguistics, especially in multicultural classrooms. This qualitative research utilized narrative interviews to explore code switching in the classroom, focusing on the teacher's narrative. Interviews were audio-recorded and supplemented with field notes. Data analysis involved narrative blocks. The research findings emphasized the effectiveness of multilingual code-switching and individualized instruction in enhancing students' English language learning. Embracing linguistic diversity, valuing students' native languages, and using creative learning approaches created a positive and inclusive environment, fostering comprehension, vocabulary retention, and cultural appreciation while motivating students. In conclusion, multilingual code-switching enhanced English learning by integrating native languages with instruction. Tailored teaching methods supported individual proficiency and learning styles, while creative code-switching boosted engagement and comprehension by linking English concepts to native languages.

Evaluation of accuracy and potential harm of ChatGPT in medical nutrition therapy - a case-based approach

Background ChatGPT is a conversational large language model (LLM) based on artificial intelligence (AI). LLMs may be applied in health care education, research, and practice if relevant valid concerns are proactively addressed. The current study aimed to investigate ChatGPT’s ability to generate accurate and comprehensive responses to nutritional queries created by nutritionists/dieticians. Methods An in-depth case study approach was used to accomplish the research objectives. Functional testing was performed, creating test cases based on the functional requirement of the software application. ChatGPT responses were evaluated and analyzed using various scenarios requiring medical nutritional therapy, which were created with varied complexity. Based on the accuracy of the generated data, which were evaluated by a registered nutritionist, a potential harm score for the responses from Chat GPT was used as evaluation. Results Eight case scenarios with varied complexity when evaluated revealed that, as the complexity of the scenario increased, it led to an increase in the risk potential. Although the accuracy of the generated response does not change much with the complexity of the case scenarios, the study suggests that ChatGPT should be avoided for generating responses for complex medical nutritional conditions or scenarios. Conclusions The need for an initiative that engages all stakeholders involved in healthcare education, research, and practice is urgently needed to set up guidelines for the responsible use of ChatGPT by healthcare educators, researchers, and practitioners. The findings of the study are useful for healthcare professionals and health technology regulators.

Extracting accurate materials data from research papers with conversational language models and prompt engineering.

There has been a growing effort to replace manual extraction of data from research papers with automated data extraction based on natural language processing, language models, and recently, large language models (LLMs). Although these methods enable efficient extraction of data from large sets of research papers, they require a significant amount of up-front effort, expertise, and coding. In this work, we propose the ChatExtract method that can fully automate very accurate data extraction with minimal initial effort and background, using an advanced conversational LLM. ChatExtract consists of a set of engineered prompts applied to a conversational LLM that both identify sentences with data, extract that data, and assure the data's correctness through a series of follow-up questions. These follow-up questions largely overcome known issues with LLMs providing factually inaccurate responses. ChatExtract can be applied with any conversational LLMs and yields very high quality data extraction. In tests on materials data, we find precision and recall both close to 90% from the best conversational LLMs, like GPT-4. We demonstrate that the exceptional performance is enabled by the information retention in a conversational model combined with purposeful redundancy and introducing uncertainty through follow-up prompts. These results suggest that approaches similar to ChatExtract, due to their simplicity, transferability, and accuracy are likely to become powerful tools for data extraction in the near future. Finally, databases for critical cooling rates of metallic glasses and yield strengths of high entropy alloys are developed using ChatExtract.

A FRENCH CONVERSATION CLUB AS A WAY TO FORM STUDENTS’ SELF-IDENTITY AT LINGUISTICS UNIVERSITIES

The article reveals the role and advantages of a foreign language conversation club as an extracurricular form of educational activity and as a way to form students’ self-identity at a linguistics university. The authors examine the concept of self-identity and its components, as well as their implementation using the case of French-language films. The conclusion is made about the advantages of this method of educational activity, which allows students to determine their public position, join a certain cognitive, cultural, psychological and social (including professional) community.

Case study regarding adult students’ learning of world languages and cultures in a unique voluntary and learner-centred nonformal setting

The purpose of this qualitative study was to examine a group of adult students’ learning of world languages and cultural practices by incorporating common literacy practices (listening, reading, speaking, writing) in a unique, voluntary, learner-centred, nonformal setting (e.g. World Language and Culture Program [WLCP]) at a large western land-grant university in the United States during the 2019 spring semester. The primary objectives of this study are to provide insight into what and how adults learned in this unique environment. Drawing from Simons, the study was framed as a qualitative descriptive case study. Data sources included interviews, observations, field notes, and artefacts. Data analysis drew on the Deep Approach conceptual lens and the Interpret Analyse Present Interact (IAPI) curriculum model to understand adult students’ learning in this WLCP. Findings showed that adults learned real-life conversation skills and cultural practices (geography, traditions, religious practices) by incorporating common literacy practices (listening, reading, speaking, writing) from volunteer facilitators who were natives of the particular world language and culture. The participants learned these world language conversation skills and cultural practices in a variety of learner-centred ways: (a) practice with heritage speakers of the language, (b) meaningful interactions with peers, (c) contextualised simulations, (d) spontaneous exposure, and (e) instructional scaffolding.