Robot Voice Research Articles

Voice‐over anatomy lectures created by AI‐voice cloning technology: A descriptive article

AbstractThe COVID‐19 pandemic revealed the pivotal role of digital learning and online lecture videos, leading a shift toward blended and flipped classrooms in the post‐pandemic era. This shift calls for development and (or) refreshment of novel online educational resources, such as Voice‐Over PowerPoint (VOPPT) presentations, specifically designed for asynchronous or synchronous learning methods. However, resource limitations often impede the timely delivery of high‐quality instructional materials. In this descriptive article, the use of AI‐voice cloning technology to automate the creation of VOPPT presentation has been explored. Descripts' Overdub tool, an AI‐voice cloning program, was trained on the 15‐min voice sample of an anatomy professor, and the synthesized voice was used to narrate presentations on inguinal canal and extrahepatic biliary anatomy. The educational use of this novel approach was evaluated based on the second‐year undergraduate medical students' qualitative feedback. Voice similarity analysis using Resemblyzer, an open‐source Python tool, showed a high similarity score of 0.92 between the cloned and original voices. Despite this, students raised concerns about the robotic voice, quick pace, punctuation problems, pronunciation difficulties, and expressed reservations about AI‐generated lecture narration. While the cloned voice closely matched the original, the AI‐generated narration fell short of capturing the nuanced details needed for an effective anatomy instruction. This uncovers AI's current limitations in the educational contexts but establishes a foundation for future progress in AI‐voice cloning technologies aimed at enhancing online educational resources for anatomy and medical education.

The Role of Name, Origin, and Voice Accent in a Robot's Ethnic Identity.

This paper presents the results of an experiment that was designed to explore whether users assigned an ethnic identity to the Misty II robot based on the robot's voice accent, place of origin, and given name. To explore this topic a 2 × 3 within subject study was run which consisted of a humanoid robot speaking with a male or female gendered voice and using three different voice accents (Chinese, American, Mexican). Using participants who identified as American, the results indicated that users were able to identify the gender and ethnic identity of the Misty II robot with a high degree of accuracy based on a minimum set of social cues. However, the version of Misty II presenting with an American ethnicity was more accurately identified than a robot presenting with cues signaling a Mexican or Chinese ethnicity. Implications of the results for the design of human-robot interfaces are discussed.

Robot voice calls asking older adults daily life difficulties for the prevention of early dementia

Robot voice calls asking older adults daily life difficulties for the prevention of early dementia

Can a virtual human increase mindfulness and reduce stress? A randomised trial

BackgroundStress is a significant issue amongst university students, yet limited psychological services are available. Mindfulness is effective for stress reduction and can be delivered digitally to expand access to student populations. However, digital interventions often suffer from low engagement and poor adherence. A virtual human may improve engagement and adherence through its humanlike appearance and behaviours. ObjectiveTo examine whether a virtual human could reduce stress in university students at least as much as a teletherapist, and more than a chatbot, using a mindfulness intervention. MethodsStressed university students (N = 158) were randomly allocated to the virtual human (N = 54), chatbot (N = 54), or teletherapist (N = 50). 36 participants received each condition. Participants completed one lab session and were asked to do online homework sessions at least twice weekly for four weeks. Changes in self-reported stress and mindfulness, physiological stress indices, homework completion, and perceptions of the agent were compared between groups. Thematic analysis was conducted on participants’ responses to open-ended questions about the interventions. ResultsThere were significant reductions in stress and increases in mindfulness across all groups. All groups had higher peripheral skin temperature post-intervention, and only the teletherapy group had higher electrodermal activity (reflecting elevated stress) post-intervention compared to baseline. There were no significant changes in heart rate. Homework adherence was significantly higher in the virtual human group, whereas homework satisfaction and engagement were lowest in the chatbot group. Thematic analysis found that people thought the robotic voice of the virtual human could be improved, the chatbot could be improved by adding audio, and that participants experienced feelings of judgement from the teletherapist. DiscussionOverall, results support use of virtual humans for delivering mindfulness interventions in stressed students. Virtual humans may have the advantage over teletherapy and chatbots of increasing adherence in student populations, but more work is needed to increase perceived empathy and replicate results in other populations.

Which Voice for which Robot? Designing Robot Voices that Indicate Robot Size

Many social robots will have the capacity to interact via speech in the future, and thus they will have to have a voice. However, so far it is unclear how we can create voices that fit their robotic speakers. In this article, we explore how robot voices can be designed to fit the size of the respective robot. We therefore investigate the acoustic correlates of human voices and body size. In Study I, we analyzed 163 speech samples in connection with their speakers’ body size and body height. Our results show that specific acoustic parameters are significantly associated with body height, and to a lesser degree to body weight, but that different features are relevant for female and male voices. In Study II, we tested then for female and male voices to what extent the acoustic features identified can be used to create voices that are reliably associated with the size of robots. The results show that the acoustic features identified provide reliable clues to whether a large or a small robot is speaking.

Media richness effectiveness: Humanoid robots with or without voice, or just a tablet kiosk?

AbstractHumanoid robots can serve as in‐store communication medium, providing functional as well as hedonic value. Previous studies have demonstrated their proficiency in providing functional value, such as guiding customers, advertising products, or carrying groceries. However, in a retailing context, engaging customers often requires more than meeting their basic functional needs. This study is the first to examine the potential of humanoid service robots (HSRs) in providing hedonic value as well, comparing their performance to that of more traditional in‐store media in entertaining customers. Based on media richness effectiveness theory, the role of robot voice is also investigated. An experimental field study was performed administering a fun quiz to passers‐by at an airport store considering three conditions (via tablet, robot with voice, or robot without voice). The results show that the quiz via a humanoid robot (as compared to via a tablet kiosk) provides customers more functional and hedonic value. While functional value favorably affects loyalty intentions mainly via enhanced store image, hedonic value has a direct and indirect effect mainly via store attitude. Adding voice to the HSR does not seem to provide significant added value. Data on actual purchase behavior also hint at superiority of the robot over a traditional tablet kiosk.

Speaking in front of cartoon avatars: A behavioral and psychophysiological study on how audience design impacts on public speaking anxiety in virtual environments

Public speaking anxiety is defined as a strong difficulty in speaking in front of an audience and has been shown to impair work performance and social relationships. Virtual Reality (VR) offers an efficient tool for modulating public speaking anxiety through a wide range of customizations concerning environmental settings. However, scientific research needs to understand better what features of the simulated environment are more important to increase or reduce participants’ perceived discomfort. The present study investigates the role of visual (human vs cartoon characters) and acoustic (human vs robotic voice) audience features on perceived anxiety, sense of presence, and perceived realism in an interactive VR public speaking scenario. 42 participants (mean age = 24 y.o; Females = 30) performed four public speaking sessions, characterized by different levels (high vs. low) of graphic and acoustic audience design. Both explicit (questionnaires) and implicit physiological measures (Electrodermal activity-EDA) collected during audience interaction were used to assess the participants’ experiences.The results showed that the features of the simulated audience played a crucial role in perceived anxiety during a virtual public speaking. Specifically, the more realistic level of graphic and acoustic stimuli resulted in higher levels of self-reported anxiety as compared to the lower realism level. However, the experienced realism and the sense of presence seem more affected by the graphical than acoustic features of the virtual environment. By contrast, the acoustic features impact on the interaction realism with the virtual audience. Interestingly, the robotic voice (lower acoustic realism) increased electrodermal response during the interaction with the audience, interpreted as a break in the sense of presence. A positive correlation between anxiety, sense of presence, and experienced realism was found. As well, perceived anxiety is correlated with electrodermal activity during the performance. Nevertheless, physiological activity is more affected by the first experience than the realism features, suggesting the presence of a habituation effect across the repeated sessions. Taken together, the results of our study showed that multisensory features (graphical and acoustic) of the virtual environment play a fundamental role in creating realistic public speaking experiences and might be used within gamification strategies for soft skill training (e.g., for improving public speaking anxiety).

Research on speech style transfer algorithm combined with image processing perspective

Speech, as the acoustic expression of language, is one of the most natural and effective means of human information communication. With the rapid development of the Internet and communication technology, the function of robot voice interaction is more and more popular among people. However, the robotic pronunciation cannot meet people's demand for personalized voice interaction. At the same time, style transfer technology, which is widely used in image and video processing, has been relatively mature. By studying the theoretical methods of generalized style transfer technology (including the style transfer of images and video signals), and comparing and analyzing various machine learning algorithms used by the current voice style transfer technology, this paper draws the following conclusions: First, various models generally have the problem of large demand for training data and difficulty in training. Second, an algorithm model shows the alienation effect in different usage scenarios. Finally, based on the above problems, suggestions for the development of voice style transfer are put forward.

The Effects of Robot Voices and Appearances on Users’ Emotion Recognition and Subjective Perception

As the influence of social robots in people’s daily lives grows, research on understanding people’s perception of robots including sociability, trust, acceptance, and preference becomes more pervasive. Research has considered visual, vocal, or tactile cues to express robots’ emotions, whereas little research has provided a holistic view in examining the interactions among different factors influencing emotion perception. We investigated multiple facets of user perception on robots during a conversational task by varying the robots’ voice types, appearances, and emotions. In our experiment, 20 participants interacted with two robots having four different voice types. While participants were reading fairy tales to the robot, the robot gave vocal feedback with seven emotions and the participants evaluated the robot’s profiles through post surveys. The results indicate that (1) the accuracy of emotion perception differed depending on presented emotions, (2) a regular human voice showed higher user preferences and naturalness, (3) but a characterized voice was more appropriate for expressing emotions with significantly higher accuracy in emotion perception, and (4) participants showed significantly higher emotion recognition accuracy with the animal robot than the humanoid robot. A follow-up study ([Formula: see text]) with voice-only conditions confirmed that the importance of embodiment. The results from this study could provide the guidelines needed to design social robots that consider emotional aspects in conversations between robots and users.

RFMic-Phone: Robust Sound Acquisition Combining Millimeter-Wave Radar and Microphone

With the increasing popularity of the Internet of Things and smart factories, robust sound acquisition plays an important role in communication and human–machine interaction, for example, in-vehicle voice control interactions, smart home, and robotic voice interaction. However, due to the inherent sensing manners of the widely used microphone system, the existing audio-only sound acquisition system has fundamental limitations in solving the cocktail problem and is difficult to adapt to complex acoustic environments. Considering the advantages of millimeter-wave (mmWave) radar in the field of target positioning and precise vibration measurement, the mmWave vibration measurement-based sound perception has great potential to solve the above problems. However, there are still difficulties in high-quality sound recovery. In this letter, we propose RFMic-Phone, a robust sound acquisition system combining a mmWave radar and a traditional microphone, which combines the sound signals captured by the mmWave radar and microphone, offering a novel approach for reliable sound acquisition in the complex acoustic environment. The microphone is used to obtain the mixed and high-fidelity audio signals, while the radar is used to obtain the vibration of the sound source. To achieve intelligent and effective feature fusion, we employ a deep learning framework with a modified convolutional encoder–decoder neural network structure. Moreover, we propose to utilize the real and imaginary spectra of the target sound source as the input of the network, allowing achieve high-quality target sound signals. The experimental results show that the RFMic-Phone can achieve robust and high-quality sound signals acquisition in a variety of complex acoustic environments.

Evaluating Users’ Auditory Affective Preference for Humanoid Robot Voices through Neural Dynamics

Users’ affective preference for voices has become a topic of great interest with the prevalence of humanoid robots. Nevertheless, the affective preference formation for humanoid voices remains unknown, and its evaluation lacks objective methods. Consequently, we conducted an EEG experiment to unravel the underlying neural dynamics and evaluate users’ affective preference for humanoid robot voices. Significantly larger P2, P3, and LPP amplitudes, enhanced theta, and decreased alpha oscillations were observed when users affectively preferred humanoid robot voices. The results suggest that the neural dynamics underlying users’ affective preference for humanoid robot voices might primarily consist of early detection of affective information in voices, further processing of affective information, and later evaluative categorization of affective preference. Moreover, the neural indicators could distinguish users’ affective preferences for humanoid robot voices. The study contributes to understanding the auditory affective preference formation for humanoid robot voices and providing a neurological evaluation method.

An Exploratory Analysis of the Neural Correlates of Human-Robot Interactions With Functional Near Infrared Spectroscopy.

Functional near infrared spectroscopy (fNIRS) has been gaining increasing interest as a practical mobile functional brain imaging technology for understanding the neural correlates of social cognition and emotional processing in the human prefrontal cortex (PFC). Considering the cognitive complexity of human-robot interactions, the aim of this study was to explore the neural correlates of emotional processing of congruent and incongruent pairs of human and robot audio-visual stimuli in the human PFC with fNIRS methodology. Hemodynamic responses from the PFC region of 29 subjects were recorded with fNIRS during an experimental paradigm which consisted of auditory and visual presentation of human and robot stimuli. Distinct neural responses to human and robot stimuli were detected at the dorsolateral prefrontal cortex (DLPFC) and orbitofrontal cortex (OFC) regions. Presentation of robot voice elicited significantly less hemodynamic response than presentation of human voice in a left OFC channel. Meanwhile, processing of human faces elicited significantly higher hemodynamic activity when compared to processing of robot faces in two left DLPFC channels and a left OFC channel. Significant correlation between the hemodynamic and behavioral responses for the face-voice mismatch effect was found in the left OFC. Our results highlight the potential of fNIRS for unraveling the neural processing of human and robot audio-visual stimuli, which might enable optimization of social robot designs and contribute to elucidation of the neural processing of human and robot stimuli in the PFC in naturalistic conditions.

Robot Voices in Daily Life: Vocal Human-Likeness and Application Context as Determinants of User Acceptance.

The growing popularity of speech interfaces goes hand in hand with the creation of synthetic voices that sound ever more human. Previous research has been inconclusive about whether anthropomorphic design features of machines are more likely to be associated with positive user responses or, conversely, with uncanny experiences. To avoid detrimental effects of synthetic voice design, it is therefore crucial to explore what level of human realism human interactors prefer and whether their evaluations may vary across different domains of application. In a randomized laboratory experiment, 165 participants listened to one of five female-sounding robot voices, each with a different degree of human realism. We assessed how much participants anthropomorphized the voice (by subjective human-likeness ratings, a name-giving task and an imagination task), how pleasant and how eerie they found it, and to what extent they would accept its use in various domains. Additionally, participants completed Big Five personality measures and a tolerance of ambiguity scale. Our results indicate a positive relationship between human-likeness and user acceptance, with the most realistic sounding voice scoring highest in pleasantness and lowest in eeriness. Participants were also more likely to assign real human names to the voice (e.g., “Julia” instead of “T380”) if it sounded more realistic. In terms of application context, participants overall indicated lower acceptance of the use of speech interfaces in social domains (care, companionship) than in others (e.g., information & navigation), though the most human-like voice was rated significantly more acceptable in social applications than the remaining four. While most personality factors did not prove influential, openness to experience was found to moderate the relationship between voice type and user acceptance such that individuals with higher openness scores rated the most human-like voice even more positively. Study results are discussed in the light of the presented theory and in relation to open research questions in the field of synthetic voice design.

Trust Me on This One: Conforming to Conversational Assistants

Conversational artificial agents and artificially intelligent (AI) voice assistants are becoming increasingly popular. Digital virtual assistants such as Siri, or conversational devices such as Amazon Echo or Google Home are permeating everyday life, and are designed to be more and more humanlike in their speech. This study investigates the effect this can have on one’s conformity with an AI assistant. In the 1950s, Solomon Asch’s already demonstrated the power and danger of conformity amongst people. In these classical experiments test persons were asked to answer relatively simple questions, whilst others pretending to be participants tried to convince the test person to give wrong answers. These studies were later replicated with embodied robots, but these physical robots are still rare. In light of our increasing reliance on AI assistants, this study investigates to what extent an individual will conform to a disembodied virtual assistant. We also investigate if there is a difference between a group that interacts with an assistant that communicates through text, one that has a robotic voice and one that has a humanlike voice. The assistant attempts to subtly influence participants’ final responses in a general knowledge quiz, and we measure how often participants change their answer after having been given advice. Results show that participants conformed significantly more often to the assistant with a human voice than the one that communicated through text.

Towards Contactless Learning Activities during Pandemics Using Autonomous Service Robots

The COVID-19 pandemic has had a significant impact worldwide, impacting schools, undergraduate, and graduate university education. More than half a million lives have been lost due to COVID-19. Moving towards contactless learning activities has become a research area due to the rapid advancement of technology, particularly in artificial intelligence and robotics. This paper proposes an autonomous service robot for handling multiple teaching assistant duties in the educational field to move towards contactless learning activities during pandemics. We use SLAM to map and navigate the environment to proctor an exam. We also propose a human–robot voice interaction and an academic content personalization algorithm. Our results show that our robot can navigate the environment to proctor students avoiding any static or dynamic obstacles. Our cheating detection system obtained a testing accuracy of 86.85%. Our image-based exam paper scanning system can scan, extract, and process exams with high accuracy.

Design of Intelligent Education Service Robots for Big Data

With the development of robot technology, service robots have gradually come into people's lives, which also makes human-computer interaction more and more frequent. However, most of the current robot control algorithms have low accuracy and difficult operation, and people can't wait to find more effective algorithms. In order to design the most convenient and accurate intelligent education service robot, this paper uses SF algorithm and improved PCNN to establish a hybrid model, and proposes an improved saliency region extraction algorithm based on education service robot. The algorithm compares the standard database with the real environment, and the PR curve of the proposed algorithm is improved by about 5%. It is 10% higher than the SF algorithm in the AUC index, and the comprehensive F value is improved by 3.4% ~ 7.4%. This paper fully demonstrates that the saliency area generated by the proposed algorithm is closer to the true value, which can effectively suppress the high-brightness background area in the detection results of the SF algorithm. It also verifies that the PCNN model with the neuron propagation stimulation mechanism as the core more effective simulation of biological vision systems. Combining compressed sensing technology, this paper proposes a speech recognition scheme that is easy to implement in hardware. The algorithm performance of the robot is verified, and the optimal effect parameters are selected through comparative experiments. The method uses Chinese phonetic phrase (sentence) test to obtain good recognition results, and it can be used as an effective improvement scheme for the speech input of the proposed robot voice interaction system.

Le robot comme « support de parole » pour l’enfant autiste. Sur l’intérêt de l’outil robotique dans la relation intersubjective enfant TSA/thérapeute

BackgroundWorking for several years in a day hospital receiving young people with ASD aged 4 to 14 years, we have introduced, since 2016, therapeutic interventions with robotic mediation. We have been able to show, in several publications, how the robot was an innovative and interesting medium in the ASD clinic, notably because of its simplified appearance, its repetitiveness and its predictability. In this article, we will focus on the simplified aspect of the robot's voice, an essential element to be taken into account in this type of therapeutic mediation. MethodBased on the case of an autistic child that we have followed in “robotic” mediation therapy, we will try to show how the robot can be a “support” or a “relay” of the subject who controls it, or in any case contain a part of the latter's own subjectivity. FindingsDavid's case confirms that autistic subjects show a great difficulty to speak about themselves, in particular through the voice. Using the robot, and more precisely the text-to-speech function of the robot Nao, allows him to communicate his feelings. The live confrontation, too distressing for David, can thus be avoided by taking a roundabout way: making the robot speak for him, and thus trying to express his feelings orally, without being too personally involved. ConclusionOne of the interests of the robot will be to be the support of an out-of-body voice, not directly addressed to the one who will be the receiver, thus allowing the autistic subject to communicate without having to go through the vibration of his body provoked by the voice, nor to be confronted with the direct meeting with another subject situated in the field of the address of the speech. Thus, even when the autistic child speaks through the machine, something is said in what is heard.

Communicative And Affective Components in Processing Auditory Vitality Forms: An fMRI Study.

In previous studies on auditory vitality forms, we found that listening to action verbs pronounced gently or rudely, produced, relative to a neutral robotic voice, activation of the dorso-central insula. One might wonder whether this insular activation depends on the conjunction of action verbs and auditory vitality forms, or whether auditory vitality forms are sufficient per se to activate the insula. To solve this issue, we presented words not related to actions such as concrete nouns (e.g.,“ball”), pronounced gently or rudely. No activation of the dorso-central insula was found. As a further step, we examined whether interjections, i.e., speech stimuli conveying communicative intention (e.g., “hello”), pronounced with different vitality forms, would be able to activate, relative to control, the insula. The results showed that stimuli conveying a communicative intention, pronounced with different auditory vitality forms activate the dorsal-central insula. These data deepen our understanding of the vitality forms processing, showing that insular activation is not specific to action verbs, but can be also activated by speech acts conveying communicative intention such as interjections. These findings also show the intrinsic social nature of vitality forms because activation of the insula was not observed in the absence of a communicative intention.

Voice-Controlled Assistant

Considering the current situation of COVID-19 this project provides medical assistance and sanitization. This project also provides a source of information as well as entertainment to the patients. Searching Google and Wikipedia based on users' voice input and giving back the reply through robotic voice with further interactive questions by the machine. It distributes the medicines i.e., morning and evening pills to the patient according to the request. According to the voice input it moves forward, backward, turns left and right. It also detects the hand through IR SENSOR & Dispenses the sanitizer to the patient.

Effect of Voice Type and Head-Light Color in Social Robots for Different Applications

The social robotics field has been growing in prominence. Many developed countries have begun to apply such robots in various fields, such as shopping reception, education, home companion, medical care, and security. However, expectations of social robots have also changed. In addition to performing their specific tasks, social robots are also expected to consider users’ emotional needs. Therefore, scholars have begun to explore robots’ social cues in an effort to enhance their role in society and enable them to provide higher-quality services. This study used the robot's voice type, head-light color, and application field as independent variables to discuss the optimal robot social cues for different applications and to provide a reference for future production and research. Education, shopping reception, and home companion applications were selected as the most common areas employing social robots. According to the vocal fundamental frequencies used in related studies, three voice types were included: male, female, and child. The three values for head-light color, namely warm, neutral, and cold colors, were set according to color temperature theory. The robot social attribute scale, with the addition of an acceptance component, was used to evaluate respondent perceptions. Results revealed that male voices provide users with the highest impression of competence, whereas children's voices have the lowest competence. However, results for the warmth component were completely different: in this aspect, children's voices had the highest evaluations and male voices the lowest. For head-light colors, neutral colors had the highest overall acceptance, the highest competence evaluation, and the lowest discomfort. The neutral color's warmth judgment was the same as that of the warm color, and its competence rating was the same as that of the cold color. For home companion robots, we recommend a child’s voice with neutral colors as the first choice, and a child’s voice with warm color as the second option. Because females rejected male voices, male voices are not recommended in home companion. For education and business, a male voice and neutral colors are the first choice. A female voice is also the second option. In cases with less focus on warmth impression but greater emphasis on competence impression, cold colors could be employed.