Conversational Language Research Articles

SVCGAN: Speaker Voice Conversion Generative Adversarial Network for Children’s Speech Conversion and Recognition

Automatic speech recognition (ASR) refers to a technological process that entails the conversion of spoken language into written text. However, the acoustic distinctions between children’s speech and adult speech are substantial, rendering the automatic speech recognition system trained on adult speech inadequate for effectively recognizing children’s speech. To overcome this issue, in this study, we propose speaker conversion generative adversarial network (SVCGAN). SVCGAN is a novel non-parallel voice conversion model, which enhances three key areas: log-cosh loss, semantic-similarity loss, and third adversarial loss. Therefore, the incorporation of these losses better protects semantic information for young children during voice conversion process and improves the quality of the converted speech. Additionally, the character error rate (CER) of children’s speech recognition can benefit from children’s speech transformed into adult speech. Experimental results suggest that SVCGAN demonstrates superior performance across multiple dimensions compared to both CycleGAN-VC3 and MaskCycleGAN-VC models. It encompasses training efficiency, semantic information similarity, voice type similarity, sound naturalness and intelligibility, which leads to a reduction in the CER of speech recognition for young children.

Enhancing Lip Reading: A Deep Learning Approach with CNN and RNN Integration

This research introduces an innovative approach to enhance lip reading-based text extraction and translation through the integration of a double Convolutional Neural Network (CNN) coupled with Recurrent Neural Network (RNN) architecture. The proposed model aims to leverage the strengths of both CNN and RNN to achieve superior accuracy in lip movement interpretation and subsequent text extraction. The methodology involves training the double CNN+RNN model on extensive datasets containing synchronized lip movements and corresponding linguistic expressions. The initial layers of the model utilize CNNs to effectively capture spatial features from the visual input of lip images. The extracted features are then fed into RNN layers, allowing the model to grasp temporal dependencies and contextual information crucial for accurate lip reading. The trained model showcases its proficiency in extracting textual content from spoken words, demonstrating an advanced capability to decipher nuances in lip gestures. Furthermore, the extracted text undergoes a translation process, enabling the conversion of spoken language into various target languages. This research not only contributes to the advancement of lip reading technologies but also establishes a robust foundation for real-world applications such as accessibility solutions for individuals with hearing impairments, real-time multilingual translation services, and improved communication in challenging acoustic environments. The abstract concludes with a discussion on the potential impact of the double CNN+RNN model in pushing the boundaries of human-computer interaction, emphasizing the synergy between deep learning, lip reading, and translation technologies

Conversational AI: An Explication of Few-Shot Learning Problem in Transformers-Based Chatbot Systems

With the recent advancements in conversational artificial intelligence (AI), the practical applications of chatbots have risen significantly in diverse domains such as healthcare, education, e-government, customer support systems, social platforms, and entertainment. The chatbot converses with humans in natural language and responds to their queries with precise and relevant answers. The relevant literature presents several methods for intent classification and slot mapping for chatbot question answering. However, the existing chatbot architectures still suffer from few-shot learning problem (imbalance ratio of class labels over samples) and ineffective dialog management to retain the context and slots mapping. This study aims to introduce the architecture of chatbot by focusing few-shot learning problem and context management in dialog-based conversations. First, to mitigate the few-shot learning problem with chatbots, we propose a novel hybrid intent and slots transformers (HIST) model. The HIST chatbot architecture utilizes transformers and self-attention mechanisms along with bigated recurrent unit and combines conditional random field algorithms for intent classification and slot extraction. Second, to address dialog management, we introduce a hybrid interaction strategy for slots mapping and effective conversational context management. To validate the proposed model’s effectiveness, comprehensive empirical analysis is carried out using three benchmark datasets including airline travel information system (ATIS), banking77, and conversational language interface for natural conversation 150 (CLINC150). The results show that HIST outperforms against the state-of-the-art existing methods with a clear margin and obtained an accuracy of 94.89% and 96.17% for intent classification and slots extraction, respectively. The empirical results confirm the effectiveness of the HIST chatbot for resolving the few-shot learning problem with effective dialog management in chatbot systems.

CAVEN: An Embodied Conversational Agent for Efficient Audio-Visual Navigation in Noisy Environments

Audio-visual navigation of an agent towards locating an audio goal is a challenging task especially when the audio is sporadic or the environment is noisy. In this paper, we present CAVEN, a Conversation-based Audio-Visual Embodied Navigation framework in which the agent may interact with a human/oracle for solving the task of navigating to an audio goal. Specifically, CAVEN is modeled as a budget-aware partially observable semi-Markov decision process that implicitly learns the uncertainty in the audio-based navigation policy to decide when and how the agent may interact with the oracle. Our CAVEN agent can engage in fully-bidirectional natural language conversations by producing relevant questions and interpret free-form, potentially noisy responses from the oracle based on the audio-visual context. To enable such a capability, CAVEN is equipped with: i) a trajectory forecasting network that is grounded in audio-visual cues to produce a potential trajectory to the estimated goal, and (ii) a natural language based question generation and reasoning network to pose an interactive question to the oracle or interpret the oracle's response to produce navigation instructions. To train the interactive modules, we present a large scale dataset: AVN-Instruct, based on the Landmark-RxR dataset. To substantiate the usefulness of conversations, we present experiments on the benchmark audio-goal task using the SoundSpaces simulator under various noisy settings. Our results reveal that our fully-conversational approach leads to nearly an order-of-magnitude improvement in success rate, especially in localizing new sound sources and against methods that use only uni-directional interaction.

FacetCRS: Multi-Faceted Preference Learning for Pricking Filter Bubbles in Conversational Recommender System

The filter bubble is a notorious issue in Recommender Systems (RSs), which describes the phenomenon whereby users are exposed to a limited and narrow range of information or content that reinforces their existing dominant preferences and beliefs. This results in a lack of exposure to diverse and varied content. Many existing works have predominantly examined filter bubbles in static or relatively-static recommendation settings. However, filter bubbles will be continuously intensified over time due to the feedback loop between the user and the system in the real-world online recommendation. To address these issues, we propose a novel paradigm, Multi-Facet Preference Learning for Pricking Filter Bubbles in Conversational Recommender System (FacetCRS), which aims to burst filter bubbles in the conversational recommender system (CRS) through timely user-item interactions via natural language conversations. By considering diverse user preferences and intentions, FacetCRS automatically model user preference into multi-facets, including entity-, word-, context-, and review-facet, to capture diverse and dynamic user preferences to prick filter bubbles in the CRS. It is an end-to-end CRS framework to adaptively learn representations of various levels of preference facet and diverse types of external knowledge. Extensive experiments on two publicly available benchmark datasets demonstrate that our proposed method achieves state-of-the-art performance in mitigating filter bubbles and enhancing recommendation quality in CRS.

Tackling Vision Language Tasks through Learning Inner Monologues

Visual language tasks such as Visual Question Answering (VQA) or Visual Entailment (VE) require AI models to comprehend and reason with both visual and textual content. Driven by the power of Large Language Models (LLMs), two prominent methods have emerged: (1) the hybrid integration between LLMs and Vision-Language Models (VLMs), where visual inputs are firstly converted into language descriptions by VLMs, serving as inputs for LLMs to generate final answer(s); (2) visual feature alignment in language space, where visual inputs are encoded as embeddings and projected to LLMs' language space via further supervised fine-tuning. The first approach provides light training costs and interpretability but is hard to be optimized in an end-to-end fashion. The second approach presents decent performance, but feature alignment usually requires large amounts of training data and lacks interpretability. To tackle this dilemma, we propose a novel approach, Inner Monologue Multi-Modal Optimization (IMMO), to solve complex vision language problems by simulating Inner Monologue, a cognitive process in which an individual engages in silent verbal communication with themselves. More specifically, we enable LLMs and VLMs to interact through natural language conversation (i.e., Inner Monologue) and propose to use a two-stage training process to learn how to do Inner Monologue (self-asking questions and answering questions). IMMO is evaluated on two popular tasks and achieves competitive performance with less training data when compared with state-of-the-art models while concurrently keeping the interpretability. The results suggest that by emulating the cognitive phenomenon of internal dialogue, our approach can enhance reasoning and explanation abilities, contributing to the more effective fusion of vision and language models. More importantly, instead of using predefined human-crafted monologues, IMMO learns this process within the deep learning models, broadening its potential applications across various AI challenges beyond vision and language tasks.

Indian Sign Language Recognition using transfer learning with Efficient Net

ABSTRACT Automatic conversion of sign language into text or speech can be really useful in communicating among deaf or mute people with no sign language knowledge. The present need is to create an automated system that can translate ISL signals into regular text and vice versa. It is advantageous for both groups to communicate their needs to one another while using publicly accessible services, such as banking, traffic signaling, and ticketing. In this manuscript, Indian Sign Language Recognition using Transfer Learning with Efficient Net (ISLR-EfficientNetB0) is proposed for Indian sign language recognition. Initially, the input images are gathered via the Indian Sign Language (ISL) dataset. The input image is pre-processing by Fairness-aware Collaborative Filtering (FACF) method. In pre-processing stage noise are removed from input images and image resizing is done. Then, Efficient Net B0 is employed to identify ISL. The proposed model is implemented on Python. Then the efficiency of the proposed approach is compared with existing models. The accuracy, precision, specificity, sensitivity, F1-score, computational time, error rate and RoC of the suggested approaches are all assessed. The proposed ISLR-EfficientNetB0 method attains 31.42%, 26.04%, and 20.45% higher accuracy, 19.24%, 26.35%, and 24.02% higher precision, 15.84%, 18.91%, and 28.16% higher specificity and 17.06%, 21.36%, and 32.47% higher sensitivity compared with existing methods, such as Indian sign language recognition under machine learning methods (ISLR-KNN), ISL recognition scheme utilizing SURF with SVM and CNN (ISLR-SVM-CNN) and Benchmarking deep neural network methods for ISL recognition (ISLR-VGG16) respectively.

Language matters for biodiversity

Abstract Language is central to the way people learn about the natural world. A salient concern of the biodiversity conservation arena has been to understand how language can be employed by scientists to communicate knowledge to nonexpert audiences and build ecological literacy. The use of analogy and narrative by scientists are prominent techniques. In this article, we consider how these two modes of language-based reasoning extend into ordinary conversational language use by the public, specifically when articulating everyday understanding and experiences of biodiversity. Drawing on a process of public engagement in a UK woodland environment, a typological framework based on principles of analogical and narrative reasoning is developed to characterize the precise character of processes of everyday biodiversity sense making. The implications of the framework are discussed in the context of future biodiversity research, particularly its participatory and educational dimensions.

The Potential Applications and Challenges of ChatGPT in the Medical Field.

ChatGPT, an AI-driven conversational large language model (LLM), has garnered significant scholarly attention since its inception, owing to its manifold applications in the realm of medical science. This study primarily examines the merits, limitations, anticipated developments, and practical applications of ChatGPT in clinical practice, healthcare, medical education, and medical research. It underscores the necessity for further research and development to enhance its performance and deployment. Moreover, future research avenues encompass ongoing enhancements and standardization of ChatGPT, mitigating its limitations, and exploring its integration and applicability in translational and personalized medicine. Reflecting the narrative nature of this review, a focused literature search was performed to identify relevant publications on ChatGPT's use in medicine. This process was aimed at gathering a broad spectrum of insights to provide a comprehensive overview of the current state and future prospects of ChatGPT in the medical domain. The objective is to aid healthcare professionals in understanding the groundbreaking advancements associated with the latest artificial intelligence tools, while also acknowledging the opportunities and challenges presented by ChatGPT.

Growth of Complex Syntax: Coordinate and Subordinate Clause Use in Elementary School-Aged Children.

Production of complex syntax is a hallmark of later language development; however, most of the research examining age-related changes has focused on adolescents or analyzed narrative language samples. Research documenting age-related changes in the production of complex syntax in elementary school-aged children in conversational language samples is limited. Therefore, the purpose of this article is to examine age-related changes in the production of coordinate and subordinate clauses in children between 5 and 10 years of age obtained from 50-utterance conversational language samples. The analytic sample included 196 children with typical language development, who ranged in age from 5;0 to 10;11 (years;months; girls = 103; boys = 96; three cases were excluded). Fifty-utterance conversational language samples were examined for use of coordinate and subordinate clauses. Results of regression analyses indicated that the production of coordinate and subordinate clauses could be predicted from age. The proportion of utterances that included subordinate clauses increased 0.20% for every month increase in age (p < .001). Coordinate clauses also continued to grow, although at a slower rate (0.10% increase for every month increase in age, p < .001). Finally, the proportion of simple utterances (i.e., utterances without coordinate or subordinate clauses) decreased with age (0.40% decrease for every month increase in age, p < .001). This study indicated that as children's age increased, they used fewer, simple, one-clause sentences and more utterances that included subordinate clauses, with or without coordinate clauses. These results were obtained from 50-utterance language samples, further supporting use of language sampling to develop intervention goals and monitor progress in therapy. https://doi.org/10.23641/asha.25262725.

MULTILINGUAL CODE-SWITCHING IN THE CLASSROOM: AN ANALYSIS FROM THE TEACHER’S NARRATIVE

Code switching, the practice of alternating between languages in conversations, was of interest to researchers in sociolinguistics, especially in multicultural classrooms. This qualitative research utilized narrative interviews to explore code switching in the classroom, focusing on the teacher's narrative. Interviews were audio-recorded and supplemented with field notes. Data analysis involved narrative blocks. The research findings emphasized the effectiveness of multilingual code-switching and individualized instruction in enhancing students' English language learning. Embracing linguistic diversity, valuing students' native languages, and using creative learning approaches created a positive and inclusive environment, fostering comprehension, vocabulary retention, and cultural appreciation while motivating students. In conclusion, multilingual code-switching enhanced English learning by integrating native languages with instruction. Tailored teaching methods supported individual proficiency and learning styles, while creative code-switching boosted engagement and comprehension by linking English concepts to native languages.

Evaluation of accuracy and potential harm of ChatGPT in medical nutrition therapy - a case-based approach

Background ChatGPT is a conversational large language model (LLM) based on artificial intelligence (AI). LLMs may be applied in health care education, research, and practice if relevant valid concerns are proactively addressed. The current study aimed to investigate ChatGPT’s ability to generate accurate and comprehensive responses to nutritional queries created by nutritionists/dieticians. Methods An in-depth case study approach was used to accomplish the research objectives. Functional testing was performed, creating test cases based on the functional requirement of the software application. ChatGPT responses were evaluated and analyzed using various scenarios requiring medical nutritional therapy, which were created with varied complexity. Based on the accuracy of the generated data, which were evaluated by a registered nutritionist, a potential harm score for the responses from Chat GPT was used as evaluation. Results Eight case scenarios with varied complexity when evaluated revealed that, as the complexity of the scenario increased, it led to an increase in the risk potential. Although the accuracy of the generated response does not change much with the complexity of the case scenarios, the study suggests that ChatGPT should be avoided for generating responses for complex medical nutritional conditions or scenarios. Conclusions The need for an initiative that engages all stakeholders involved in healthcare education, research, and practice is urgently needed to set up guidelines for the responsible use of ChatGPT by healthcare educators, researchers, and practitioners. The findings of the study are useful for healthcare professionals and health technology regulators.

Extracting accurate materials data from research papers with conversational language models and prompt engineering.

There has been a growing effort to replace manual extraction of data from research papers with automated data extraction based on natural language processing, language models, and recently, large language models (LLMs). Although these methods enable efficient extraction of data from large sets of research papers, they require a significant amount of up-front effort, expertise, and coding. In this work, we propose the ChatExtract method that can fully automate very accurate data extraction with minimal initial effort and background, using an advanced conversational LLM. ChatExtract consists of a set of engineered prompts applied to a conversational LLM that both identify sentences with data, extract that data, and assure the data's correctness through a series of follow-up questions. These follow-up questions largely overcome known issues with LLMs providing factually inaccurate responses. ChatExtract can be applied with any conversational LLMs and yields very high quality data extraction. In tests on materials data, we find precision and recall both close to 90% from the best conversational LLMs, like GPT-4. We demonstrate that the exceptional performance is enabled by the information retention in a conversational model combined with purposeful redundancy and introducing uncertainty through follow-up prompts. These results suggest that approaches similar to ChatExtract, due to their simplicity, transferability, and accuracy are likely to become powerful tools for data extraction in the near future. Finally, databases for critical cooling rates of metallic glasses and yield strengths of high entropy alloys are developed using ChatExtract.

A FRENCH CONVERSATION CLUB AS A WAY TO FORM STUDENTS’ SELF-IDENTITY AT LINGUISTICS UNIVERSITIES

The article reveals the role and advantages of a foreign language conversation club as an extracurricular form of educational activity and as a way to form students’ self-identity at a linguistics university. The authors examine the concept of self-identity and its components, as well as their implementation using the case of French-language films. The conclusion is made about the advantages of this method of educational activity, which allows students to determine their public position, join a certain cognitive, cultural, psychological and social (including professional) community.

Case study regarding adult students’ learning of world languages and cultures in a unique voluntary and learner-centred nonformal setting

The purpose of this qualitative study was to examine a group of adult students’ learning of world languages and cultural practices by incorporating common literacy practices (listening, reading, speaking, writing) in a unique, voluntary, learner-centred, nonformal setting (e.g. World Language and Culture Program [WLCP]) at a large western land-grant university in the United States during the 2019 spring semester. The primary objectives of this study are to provide insight into what and how adults learned in this unique environment. Drawing from Simons, the study was framed as a qualitative descriptive case study. Data sources included interviews, observations, field notes, and artefacts. Data analysis drew on the Deep Approach conceptual lens and the Interpret Analyse Present Interact (IAPI) curriculum model to understand adult students’ learning in this WLCP. Findings showed that adults learned real-life conversation skills and cultural practices (geography, traditions, religious practices) by incorporating common literacy practices (listening, reading, speaking, writing) from volunteer facilitators who were natives of the particular world language and culture. The participants learned these world language conversation skills and cultural practices in a variety of learner-centred ways: (a) practice with heritage speakers of the language, (b) meaningful interactions with peers, (c) contextualised simulations, (d) spontaneous exposure, and (e) instructional scaffolding.

Role and challenges of ChatGPT and similar generative artificial intelligence in arts and humanities

In the dynamic world of technology, ChatGPT has emerged as a transformative tool within the realm of Arts and Humanities, facilitating innovative interactions between humans and machines. This research delves into the multifaceted role of ChatGPT in the Arts and Humanities, illuminating its potential applications and the challenges it presents. ChatGPT serves as a creative collaborator, providing artists and humanists with a distinctive platform for ideation, brainstorming, and refining their artistic expressions. Through natural language conversations, it stimulates creative thinking processes, offering valuable insights and diverse perspectives. In the realm of Humanities, ChatGPT acts as a crucial research assistant, supporting scholars in tasks like data analysis, literature review, and information synthesis. Its ability to process vast textual data expedites research, allowing scholars to delve deeper into their inquiries. However, integrating ChatGPT into the Arts and Humanities domain is not without its challenges. Ethical considerations, such as data privacy and bias, require careful attention. Balancing the augmentation of human creativity with the preservation of genuine artistic expression is a complex task. Additionally, ensuring accessibility and inclusivity in ChatGPT-generated content remains a concern, underscoring the need for continuous refinement and awareness. This research explores the evolving landscape of Arts and Humanities, where ChatGPT acts as a catalyst for creativity, critical thinking, and scholarly endeavors. By addressing these challenges and harnessing its potential, ChatGPT stands poised to bridge the gap between technology and the intricate nuances of human creativity in unprecedented ways.

The Role of Research Obsession in Shifting ChatGPT into a Plagiarism Tool

This research aims to investigate the dynamic landscape of artificial intelligence, with a particular focus on the consequences of the extensive utilization of ChatGPT's which participates in transformation it from a conversational language model to a potential tool that facilitates plagiarism for research purposes because of research obsession. As the digital age advances and with the increasing prevalence of academic dishonesty and content theft, there is a growing need for sophisticated tools to identify and fight plagiarism. As a result, the concerns about the ethical use of artificial intelligence are becoming increasingly important. This article examines the impact of research obsession on the development of ChatGPT and the unintended consequences that led to its misuse as a plagiarism tool. This study aims too to explore the underlying factors that have a part in this change, the ethical implications, and the possible remedies that can be used to help in to responsible AI development. The study hopes to contribute to the advancement of the discussion about plagiarism that depends on ChatGPT and enhance discussions on the responsible and ethical use of such technologies. By debating the effect of the research obsession on shifting ChatGPT to a tool for plagiarism, the study hopes to open new avenues for enhancing the integrity of academic and creative works in the digital era.

Research on Yulin Historical and Cultural Neighborhoods under the Threshold of Ecological Translation Studies and its Countermeasures for Public Notice Translation

With the continuous development of globalization, the importance of translation, as an important means of cultural exchange, is becoming more and more prominent. Ecological Translation Studies provides a new perspective to understand and practice translation activities. As an important cultural heritage in northern China, the translation of public notices in Yulin Historical and Cultural Neighborhood is not only a task of language conversion, but also a process of cultural transmission and adaptation. The purpose of this paper is to explore how to effectively deal with the translation problems in this specific cultural environment under the framework of ecological translation studies, and to propose practical translation countermeasures.

ChatGPT: a reliable fertility decision-making tool?

The internet is the primary source of infertility-related information for most people who are experiencing fertility issues. Although no longer shrouded in stigma, the privacy of interacting only with a computer provides a sense of safety when engaging with sensitive content and allows for diverse and geographically dispersed communities to connect and share their experiences. It also provides businesses with a virtual marketplace for their products. The introduction of ChatGPT, a conversational language model developed by OpenAI to understand and generate human-like text in response to user input, in November 2022, and other emerging generative artificial intelligence (AI) language models, has changed and will continue to change the way we interact with large volumes of digital information. When it comes to its application in health information seeking, specifically in relation to fertility in this case, is ChatGPT a friend or foe in helping people make well-informed decisions? Furthermore, if deemed useful, how can we ensure this technology supports fertility-related decision-making? After conducting a study into the quality of the information provided by ChatGPT to people seeking information on fertility, we explore the potential benefits and pitfalls of using generative AI as a tool to support decision-making.

Stylistic Conversion and Linguistic Beauty Reproduction in English Translation-Based on Neurosemantic Analysis

Abstract The core of translation is language conversion, but it should be more of an aesthetic experience, a transformation of different ways of thinking. This paper proposes a neural semantic analysis method for English translation based on text similarity calculation. The technique uses a large-scale corpus to train the vector space model of words, establishes a concept-context matrix through the term-document matrix, and determines each value of the concept vector with a weight according to the literary translation in the knowledge base. The information extraction and evaluation of the English-translated text through semantic similarity calculation provides a theoretical basis for the subsequent analysis of style conversion and linguistic, and aesthetic reproduction of the English-translated text. The study will take the Lin Yutang translation and the Ricardian translation of The Analects of Confucius as an example to develop the analysis of style transformation and reproduction of linguistic beauty of the English-translated text from both lexical and syntactic levels. The results show that the lexical density of Lin Yutang’s translation is 0.5866, while that of Rie Jacob’s translation is 0.5334, with the former significantly larger than the latter. It shows that Lin Yutang’s translation of The Analects of Confucius adopts a naturalization strategy, i.e., To disseminate Chinese culture in a way that is authentic, familiar to English readers, and easy to understand, so as to make his linguistic expression more emotional.