Audiovisual Cues Research Articles

Investigating the impacts of auditory and visual feedback in advanced driver assistance systems: a pilot study on driver behavior and emotional response

In the autonomous vehicle industry, Advanced Driver Assistance Systems (ADAS) are recognized for their capacity to enhance service quality, improve on-road safety, and increase driver comfort. Driver Assistance Systems are able to provide multi-modal feedback including auditory cues, visual cues, vibrotactile cues and so on. The study will concentrate on assessing the impacts of auditory and visual feedback from assistive driving systems on drivers. A group consisting of five participants (N = 5) was recruited to take part in two sets of driving experiments. During the experimental sessions, they were exposed to several reminders designed for drivers in audio-only format and audio-visual format, respectively. Their driving behaviors and performances were under researcher’s observation, while their emotions were evaluated by YOLO v5 detecting model. The results reveal that the participants higher compliance rate and strong emotional reactions (especially the feelings of anger, sadness and surprise) towards the unimodal feedback of audio-only driving reminders. There is no strong evidence showing that the bimodal ADAS feedback of audio-visual cues effectively improve drivers’ performance during driving period. However, both the emotion data and user satisfaction results indicate that participants experienced an increase in feelings of happiness when they were able to visualize the AI assistant while hearing the audio reminders from the assistant. The study serves as one of the pioneering studies aimed at enhancing the theoretical foundation in the field of automotive user interface design, particularly concerning the design of auditory functions.

Minimally disruptive auditory cues: their impact on visual performance in virtual reality

AbstractVirtual reality (VR) has the potential to become a revolutionary technology with a significant impact on our daily lives. The immersive experience provided by VR equipment, where the user’s body and senses are used to interact with the surrounding content, accompanied by the feeling of presence elicits a realistic behavioral response. In this work, we leverage the full control of audiovisual cues provided by VR to study an audiovisual suppression effect (ASE) where auditory stimuli degrade visual performance. In particular, we explore if barely audible sounds (in the range of the limits of hearing frequencies) generated following a specific spatiotemporal setup can still trigger the ASE while participants are experiencing high cognitive loads. A first study is carried out to find out how sound volume and frequency can impact this suppression effect, while the second study includes higher cognitive load scenarios closer to real applications. Our results show that the ASE is robust to variations in frequency, volume and cognitive load, achieving a reduction of visual perception with the proposed hardly audible sounds. Using such auditory cues means that this effect could be used in real applications, from entertaining to VR techniques like redirected walking.

Move, React, Repeat! The Role of Continuous Cues in Immersive Exergames

Immersive virtual reality (VR) exergames blend playful environments with physical activities. In such games, providing continuous cues, i.e., real-time indications of how the movements are being performed, can guide players and improve the execution of movements. However, research about how continuous cues impact player performance and experience in VR exergames remains sparse. To this end, we conducted a within-participants study (N=32) with four conditions: (i) no, (ii) audio, (iii) visual, and (iv) audiovisual cueing. The results show that both unimodal and bimodal cueing improve the player experience without causing cybersickness, while bimodal cueing improves pose accuracy. We provide three main design implications for VR exergames: (i) considering the granularity of cues, (ii) exploring different continuous cue designs, and (iii) underlining the use of continuous cues in other immersive experiences.

Audiovisual speech perception of multilingual learners of Japanese

ABSTRACT Due to the complex nature of languages and the way they are used, it is often difficult to research multilinguals because of the considerable variation in the ways their spoken languages may impact their perceptions. For this reason, foreign language education research has largely been focused on English L2 speakers, overlooking how different languages might impact perception in multilingual speakers. Focusing on Japanese as a Second Language (JSL) learners, this research investigated how multilinguals responded to audiovisual cues in Japanese and English. Utilising web-based eye-tracking technology facilitated by WebGazer.js. Participants were asked to complete a novel phonetic distinction task while their gaze behaviour was measured. Contrary to initial theoretical expectations, the findings did not reveal significant mediation by gaze behaviour on the relationship between multilingualism and task performance. However, the study found that when looking at the duration of fixation on the mouth, there's a significant difference between native English speakers and native speakers of Asian languages when listening to English or Japanese. These findings indicate that further research that explores the nuances of gaze behaviour among multilingual learners of non-English languages (such as Japanese) is necessary.

Human detection of political speech deepfakes across transcripts, audio, and video

Recent advances in technology for hyper-realistic visual and audio effects provoke the concern that deepfake videos of political speeches will soon be indistinguishable from authentic video. We conduct 5 pre-registered randomized experiments with N = 2215 participants to evaluate how accurately humans distinguish real political speeches from fabrications across base rates of misinformation, audio sources, question framings with and without priming, and media modalities. We do not find base rates of misinformation have statistically significant effects on discernment. We find deepfakes with audio produced by the state-of-the-art text-to-speech algorithms are harder to discern than the same deepfakes with voice actor audio. Moreover across all experiments and question framings, we find audio and visual information enables more accurate discernment than text alone: human discernment relies more on how something is said, the audio-visual cues, than what is said, the speech content.

Subpopulations of neurons in the perirhinal cortex enable both modality-specific and modality-invariant recognition of objects.

The perirhinal cortex (PER) supports multimodal object recognition, but how multimodal information of objects is integrated within the PER remains unknown. Here, we recorded single units within the PER while rats performed a PER-dependent multimodal object-recognition task. In this task, audiovisual cues were presented simultaneously (multimodally) or separately (unimodally). We identified 2 types of object-selective neurons in the PER: crossmodal cells, showing constant firing patterns for an object irrespective of its modality, and unimodal cells, showing a preference for a specific modality. Unimodal cells further dissociated unimodal and multimodal versions of the object by modulating their firing rates according to the modality condition. A population-decoding analysis confirmed that the PER could perform both modality-invariant and modality-specific object decoding-the former for recognizing an object as the same in various conditions and the latter for remembering modality-specific experiences of the same object.

Normoacusia y discriminación verbal en entornos sonoros reales

IntroductionHuman beings are constantly exposed to complex acoustic environments every day, which even pose challenges for individuals with normal hearing. Speech perception relies not only on fixed elements within the acoustic wave but is also influenced by various factors. These factors include speech intensity, environmental noise, the presence of other speakers, individual specific characteristics, spatial separatios of sound sources, ambient reverberation, and audiovisual cues. The objective of this study is twofold: to determine the auditory capacity of normal hearing individuals to discriminate spoken words in real-life acoustic conditions and perform a phonetic analysis of misunderstood spoken words. Materials and methodsThis is a descriptive observational cross-sectional study involving 20 normal hearing individuals. Verbal audiometry was conducted in an open-field environment, with sounds masked by simulated real-word acoustic environment at various sound intensity levels. To enhance sound emission, 2D visual images related to the sounds were displayed on a television. We analyzed the percentage of correct answers and performed a phonetic analysis of misunderstood spanish bisyllabic words in each environment. Results14 women (70%) and 6 men (30%), with an average age of 26±5,4 years and a mean airway hearing threshold in the right ear of 10,56±3,52dB SPL and in the left ear of 10,12±2,49dB SPL. The percentage of verbal discrimination in the ‘Ocean’ sound environment was 97,2±5,04%, ‘Restaurant’ was 94±4,58%, and ‘Traffic’ was 86,2±9,94% (p=0,000). Regarding the phonetic analysis, the allophones that exhibited statistically significant differences were as follows: [o] (p=0,002) within the group of vocalic phonemes, [n] (p=0,000) of voiced nasal consonants, [ɾ] (p=0,0016) of voiced fricatives, [b] (p=0,000) and [g] (p=0,045) of voiced stops. ConclusionsThe dynamic properties of the acoustic environment can impact the ability of a normal hearing individual to extract information from a voice signal. Our study demonstrates that this ability decreases when the voice signal is masked by one or more simultaneous interfering voices, as observed in a ‘Restaurant’ environment, and when it is masked by a continuous and intense noise environment such as ‘Traffic’. Regarding the phonetic analysis, when the sound environment was composed of continuous-low frequency noise, we found that nasal consonants were particularly challenging to identify. Furthermore, in situations with distracting verbal signals, vowels and vibrating consonants exhibited the worst intelligibility.

Normal hearing and verbal discrimination in real sounds environments

IntroductionHuman beings are constantly exposed to complex acoustic environments every day, which even pose challenges for individuals with normal hearing. Speech perception relies not only on fixed elements within the acoustic wave but is also influenced by various factors. These factors include speech intensity, environmental noise, the presence of other speakers, individual specific characteristics, spatial separatios of sound sources, ambient reverberation, and audiovisual cues. The objective of this study is twofold: to determine the auditory capacity of normal hearing individuals to discriminate spoken words in real-life acoustic conditions and perform a phonetic analysis of misunderstood spoken words. Materials and methodsThis is a descriptive observational cross-sectional study involving 20 normal hearing individuals. Verbal audiometry was conducted in an open-field environment, with sounds masked by simulated real-word acoustic environment at various sound intensity levels. To enhance sound emission, 2D visual images related to the sounds were displayed on a television. We analyzed the percentage of correct answers and performed a phonetic analysis of misunderstood Spanish bisyllabic words in each environment. Results14 women (70%) and 6 men (30%), with an average age of 26 ± 5,4 years and a mean airway hearing threshold in the right ear of 10,56 ± 3,52 dB SPL and in the left ear of 10,12 ± 2,49 dB SPL. The percentage of verbal discrimination in the “Ocean” sound environment was 97,2 ± 5,04%, “Restaurant” was 94 ± 4,58%, and “Traffic” was 86,2 ± 9,94% (p = 0,000). Regarding the phonetic analysis, the allophones that exhibited statistically significant differences were as follows: [o] (p = 0,002) within the group of vocalic phonemes, [n] (p = 0,000) of voiced nasal consonants, [r] (p = 0,0016) of voiced fricatives, [b] (p = 0,000) and [g] (p = 0,045) of voiced stops. ConclusionThe dynamic properties of the acoustic environment can impact the ability of a normal hearing individual to extract information from a voice signal. Our study demonstrates that this ability decreases when the voice signal is masked by one or more simultaneous interfering voices, as observed in a “Restaurant” environment, and when it is masked by a continuous and intense noise environment such as “Traffic”. Regarding the phonetic analysis, when the sound environment was composed of continuous-low frequency noise, we found that nasal consonants were particularly challenging to identify. Furthermore in situations with distracting verbal signals, vowels and vibrating consonants exhibited the worst intelligibility.

CAVEN: An Embodied Conversational Agent for Efficient Audio-Visual Navigation in Noisy Environments

Audio-visual navigation of an agent towards locating an audio goal is a challenging task especially when the audio is sporadic or the environment is noisy. In this paper, we present CAVEN, a Conversation-based Audio-Visual Embodied Navigation framework in which the agent may interact with a human/oracle for solving the task of navigating to an audio goal. Specifically, CAVEN is modeled as a budget-aware partially observable semi-Markov decision process that implicitly learns the uncertainty in the audio-based navigation policy to decide when and how the agent may interact with the oracle. Our CAVEN agent can engage in fully-bidirectional natural language conversations by producing relevant questions and interpret free-form, potentially noisy responses from the oracle based on the audio-visual context. To enable such a capability, CAVEN is equipped with: i) a trajectory forecasting network that is grounded in audio-visual cues to produce a potential trajectory to the estimated goal, and (ii) a natural language based question generation and reasoning network to pose an interactive question to the oracle or interpret the oracle's response to produce navigation instructions. To train the interactive modules, we present a large scale dataset: AVN-Instruct, based on the Landmark-RxR dataset. To substantiate the usefulness of conversations, we present experiments on the benchmark audio-goal task using the SoundSpaces simulator under various noisy settings. Our results reveal that our fully-conversational approach leads to nearly an order-of-magnitude improvement in success rate, especially in localizing new sound sources and against methods that use only uni-directional interaction.

The potency of audiovisual attractiveness and influencer marketing: The road to customer behavioral engagement

Digital technologies have signified audiovisual (vs. textual) cues for customers on online booking platforms (e.g., Airbnb and hotels). Predicating on the current literature, this study investigates the influencers' audiovisual attractiveness, customer-influencer interaction, and their consequent customer behavioral engagement using three experimental studies. This research involved an analysis using IBM SPSS 28.0, which included independent sample t-test and One-way ANOVA. Additionally, moderation analysis was conducted using the PROCESS macro model 4. In order, Study 1 reveals that influencers' audiovisual attractiveness at high (vs. low) significantly and directly manipulate customer engagement. In Study 2, using Process Model 4, customer-influencer interaction significantly mediates the relationships between influencers' audiovisual attractiveness and customer behavioral engagement. Last, Study 3 incorporates the moderation of influencers' audiovisual expertise between customer-influencer interaction and customer behavioral engagement. The two-way between-subjects analysis anchors that when influencers’ audiovisual expertise is high (vs. low), and customers having good (vs. bad) interactions, it engenders higher effects on customer behavioral engagement. The findings add to the literature on consumer engagement on booking platforms and underscore the potency of audiovisual attractiveness to engage customers. We also acknowledge the shortcomings and provide recommendations for future studies.

Dopamine activity in the nigrostriatal pathway alters cue-induced risky choice patterns in female rats.

Deficits in cost/benefit decision making is a critical risk factor for gambling disorder. Reward-paired cues may play an important role, as these stimuli can enhance risk preference in rats. Despite extensive research implicating the dorsal striatum in the compulsive aspects of addiction, the role of nigrostriatal dopaminergic activity in cue-induced risk preference remains unclear, particularly in females. Accordingly, we examined the effects of manipulating the dopaminergic nigrostriatal pathway on cue-induced risky choice in female rats. TH:Cre rats were trained on the cued version of the rat Gambling Task. This task was designed such that maximal reward is attained by avoiding the high-risk, high-reward options and instead favouring the options associated with lower per-trial gains, as they feature less frequent and shorter time-out penalties. Adding reward-paired audiovisual cues to the task leads to greater risky choice on average. To assess the role of the nigrostriatal pathway, a viral vector carrying either Cre-dependent inhibitory or excitatory DREADD was infused into the substantia nigra. Rats then received clozapine-N-oxide either during task acquisition or after a stable performance baseline was reached. Inhibition of this pathway accelerated the development of risk preference in early sessions and increased risky choice during performance, but long-term inhibition actually improved decision making. Activation of this pathway had minimal effects. These results provide evidence for the involvement of the dopaminergic nigrostriatal pathway in cue-induced risk preference in females, therefore shedding light on its role in cost/benefit decision-making deficits and expanding our knowledge of the female dopaminergic system.

A study on the effect of different channel cues on learning in immersive 360° videos.

Immersive 360° videos are of interest to educators because of their ability to provide immersive sensory experience and other features. This study examined the effects of four cue conditions on 360° video learning performance, attention, cognitive load, and mood using eye-tracking devices, brainwave meters, and subjective questionnaires. The randomly assigned participants (n = 62) did go to the experimental group (visual cues only, auditory cues only, and audiovisual cues) or the control group (no cues). The results showed that visual and audiovisual cues effectively guide learners' attention to the related learning content, reduce cognitive load during learning, and improve retention performance but have no significant effect on knowledge transfer or long-term memory. Auditory cues increase the number of times learners look at the related learning content but do not affect gaze duration and distract their attention, hindering the acquisition of relevant learning content. The study also found that visual cues effectively increase the number of times learners looked at the content. However, they do not affect gaze duration. The study also revealed that visual cues effectively increase learners' relaxation when viewing 360° videos. The study's findings can provide a reference for the instructional processing of information related to 360° video design and its practical application in teaching.

Propensity for risky choices despite lower cue reactivity in adolescent rats.

Adolescence is a time of heightened risk-taking across species. Salient audiovisual cues associated with rewards are a common feature of gambling environments and have been connected to increased risky decision-making. We have previously shown that, in adult male rats, sign tracking - a behavioral measure of cue reactivity - predicts an individual's propensity for suboptimal risky choices in a rodent gambling task (rGT) with win-paired cues. However, adolescents perform less sign tracking than adult animals, suggesting that they are less cue-reactive than adults in some circumstances. Therefore, we investigated the performance of adolescent male rats on the rGT with win cues and examined its relationship with their sign-tracking behavior. We found that adolescents make more risky choices and fewer optimal choices on the rGT compared with adults, evidence of the validity of the rGT as a model of adolescent gambling behavior. We also confirmed that adolescents perform less sign tracking than adults, and we found that, unlike in adults, adolescents' sign tracking was unrelated to their risk-taking in the rGT. This implies that adolescent risk-taking is less likely than that of adults to be driven by reward-related cues. Finally, we found that adults trained on the rGT as adolescents retained an adolescent-like propensity toward risky choices, suggesting that early exposure to a gambling environment may have a long-lasting impact on risk-taking behavior.

Exploring audiovisual speech perception in monolingual and bilingual children in Uzbekistan

This study aimed to investigate the development of audiovisual speech perception in monolingual Uzbek-speaking and bilingual Uzbek–Russian-speaking children, focusing on the impact of language experience on audiovisual speech perception and the role of visual phonetic (i.e., mouth movements corresponding to phonetic/lexical information) and temporal (i.e., timing of speech signals) cues. A total of 321 children aged 4 to 10 years in Tashkent, Uzbekistan, discriminated /ba/ and /da/ syllables across three conditions: auditory-only, audiovisual phonetic (i.e., sound accompanied by mouth movements), and audiovisual temporal (i.e., sound onset/offset accompanied by mouth opening/closing). Effects of modality (audiovisual phonetic, audiovisual temporal, or audio-only cues), age, group (monolingual or bilingual), and their interactions were tested using a Bayesian regression model. Overall, older participants performed better than younger participants. Participants performed better in the audiovisual phonetic modality compared with the auditory modality. However, no significant difference between monolingual and bilingual children was observed across all modalities. This finding stands in contrast to earlier studies. We attribute the contrasting findings of our study and the existing literature to the cross-linguistic similarity of the language pairs involved. When the languages spoken by bilinguals exhibit substantial linguistic similarity, there may be an increased necessity to disambiguate speech signals, leading to a greater reliance on audiovisual cues. The limited phonological similarity between Uzbek and Russian might have minimized bilinguals’ need to rely on visual speech cues, contributing to the lack of group differences in our study.

A101Adult Critical Transfer Simulation: Technology In Motion

Transfer medicine is evolving into a more visible, necessary sub-specialty, particularly since the COVID-19 pandemic, during which the need for proficiency in critical care transfers became apparent. As such, there exists a requirement for the provision of good training in this domain, especially for effective simulation-based learning given the dynamic nature of critical transfers, in tandem with the relative scarcity of experience of the average non-specialist practitioner. The creation of meaningful simulation training is already well recognized as posing numerous challenges [1], often in balancing fidelity with practicality, for achieving engagement of learners and transfer of learning to practice [2]. When adding the dynamism involved in a transfer - such that simulation requires movement of a critically monitored patient from one point to another - this task becomes more complex. Importantly, the integrity of the exercise is lost when provided in a stationary environment, as with much simulation traditionally. This was the focus when designing the Adult Critical Transfer Simulation (ACTS) course - a multi-disciplinary team-based day with pre-course e-learning, focused around assorted simulated scenarios performed in motion across an education suite. Additionally, it incorporates reflexive debriefing, interactive workshops and group discussion sessions to support learning. The greatest technical challenge of ACTS - modified repeatedly now over two years - was the delivery of mobile, multi-environment scenarios that provide an equivocal experience to that of traditional, stationary lab-based simulation. This was addressed utilizing wireless, remotely controlled patient models and monitoring alongside a network of strategically placed cameras and microphones to provide immersive simulation for both active and observing participants. In addition, considerable attention was given to audio-visual cues at all points of transition for creating the ambience of a moving transfer, the efficacy of which is reflected in positive course feedback. Feedback on ACTS was collated from all participants, with significantly favourable responses in every domain and 100% overall score of 5/5 for both enjoyment and recommendation, and 93.8% score of 5/5 for relevance to professional needs ( Feedback from participants of the 2023 Winter run Although there is no substitute for real-life experience, ACTS has demonstrated multi-disciplinary need for critical transfer proficiency can be met successfully through the provision of dynamic simulation when facilitated by technology, with great scope for future development given the rapidly advancing nature of technological resources. Authors confirm that all relevant ethical standards for research conduct and dissemination have been met. The submitting author confirms that relevant ethical approval was granted, if applicable.

The influence of temporal (a)synchrony and linguistic complexity on audiovisual speech perception

Audiovisual (AV) cues often lead to improved speech understanding, especially when the auditory input is impoverished (e.g., in background noise, for people with hearing loss). This AV improvement occurs when both the auditory and visual information can be integrated together effectively (e.g., through temporal synchrony). However, whether the complexity of speech signals influences this integrated percept and how this contributes to speech understanding is still unknown. This study investigates whether individual differences in perception of temporal synchrony of the auditory and visual speech signal explain individual variability in AV speech recognition, and whether this differs across different levels of linguistic processing. By implementing five different levels of linguistic complexity (i.e., varying phoneme-viseme connections, vocabulary knowledge and/or linguistic context) and introducing temporal asynchrony in a remote task with English speaking adults we ask the following questions: (1) Is temporal synchrony needed to observe AV speech perception benefits? (2) Does this vary by linguistic level? (3) Does linguistic complexity influence synchrony perception? (4) And does synchrony perception explain individual variability in AV speech perception? These findings add value in deciding how AV synchronization should be balanced in hearing aid designs as well as in augmented / virtual reality devices.

A Prospective Randomized Controlled Pilot Simulation Study to Investigate the Effect of Audiovisual Decision Support on Diagnosis and Therapeutic Interventions.

Combining audiovisual decision support during perioperative critical events might enhance provider diagnostic and therapeutic accuracy and efficiency. This study is a prospective, randomized controlled pilot trial studying the impact of audiovisual decision support on anesthesia professional performance at NorthShore University HealthSystem's high fidelity simulation center. Twenty anesthesia professionals (>2 years of clinical experience in the current role) were randomized to 2 groups (current care model vs. audiovisual assistance) and underwent 3 periprocedural simulation scenarios, where patient deterioration occurs: anaphylaxis, amniotic fluid embolism, and cardiac arrest during dental case. Overall, there was a statistically significant decrease in the mean and median pooled times to diagnosis in both the amniotic fluid embolism and pediatric dental scenarios. There was a statistically significant increase in the number of participants in the intervention group who made diagnosis 3 before the end of the scene ( P = 0.03) in the amniotic fluid embolism case. In the pediatric dental case, there was a statistically significant reduction in the median time to diagnosis 1 and diagnosis 3 in the intervention group versus control ( P = 0.01 and P = 0.0002). A significant increase in the number of participants in the intervention group versus control made the correct diagnosis 2 before vital sign change 3 ( P = 0.03), and more participants in the intervention group made the correct diagnosis 3 before the end of the scene when compared with control ( P = 0.001). The median time to start intervention 2 during the dental case was statistically significantly greater in the intervention group versus the control ( P = 0.05). All other endpoints were not statistically significant among the 3 simulation scenarios. Six questions were answered by all participants upon immediate completion of the simulation scenarios and revealed that 19 of 20 participants had delivered anesthesia care to patients similar to the 3 simulation scenarios and 18 of 20 participants reported that they would prefer audiovisual assistance to detect abnormalities in vital signs that subsequently provides appropriate diagnostic and therapeutic options. This pilot study suggested some significant improvement in anesthesia professional time to correct diagnosis and completion of identification of the correct diagnosis before the next vital change in the audiovisual cue group versus control, particularly in the outpatient dental case. In addition, the mean and median pooled times to diagnosis were significantly reduced by approximately 1 minute in both evaluated simulation scenarios. The postsimulation survey responses also suggest the desirability of an audiovisual decision support tool among the current anesthesia professional participants. However, overall, there were no significant differences in the time to intervention between groups in all simulation scenarios.

Word Learning in Deaf Adults Who Use Cochlear Implants: The Role of Talker Variability and Attention to the Mouth.

Although cochlear implants (CIs) facilitate spoken language acquisition, many CI listeners experience difficulty learning new words. Studies have shown that highly variable stimulus input and audiovisual cues improve speech perception in CI listeners. However, less is known whether these two factors improve perception in a word learning context. Furthermore, few studies have examined how CI listeners direct their gaze to efficiently capture visual information available on a talker's face. The purpose of this study was two-fold: (1) to examine whether talker variability could improve word learning in CI listeners and (2) to examine how CI listeners direct their gaze while viewing a talker speak. Eighteen adults with CIs and 10 adults with normal hearing (NH) learned eight novel word-object pairs spoken by a single talker or six different talkers (multiple talkers). The word learning task comprised of nonsense words following the phonotactic rules of English. Learning was probed using a novel talker in a two-alternative forced-choice eye gaze task. Learners' eye movements to the mouth and the target object (accuracy) were tracked over time. Both groups performed near ceiling during the test phase, regardless of whether they learned from the same talker or different talkers. However, compared to listeners with NH, CI listeners directed their gaze significantly more to the talker's mouth while learning the words. Unlike NH listeners who can successfully learn words without focusing on the talker's mouth, CI listeners tended to direct their gaze to the talker's mouth, which may facilitate learning. This finding is consistent with the hypothesis that CI listeners use a visual processing strategy that efficiently captures redundant audiovisual speech cues available at the mouth. Due to ceiling effects, however, it is unclear whether talker variability facilitated word learning for adult CI listeners, an issue that should be addressed in future work using more difficult listening conditions.

Development and Evaluation of a Mixed-Reality Tele-ultrasound System.

The objective of this feasibility study was to develop and assess a tele-ultrasound system that would enable an expert sonographer (situated at the remote site) to provide real-time guidance to an operator (situated at the imaging site) using a mixed-reality environment. An architecture along with the operational workflow of the system is designed and a prototype is developed that enables guidance in form of audiovisual cues. The visual cues comprise holograms (of the ultrasound images and ultrasound probe) and is rendered to the operator using a head-mounted display device. The position and orientation of the ultrasound probe's hologram are remotely controlled by the expert sonographer and guide the placement of a physical ultrasound probe at the imaging site. The developed prototype was evaluated for its performance on a network. In addition, a user study (with 12 participants) was conducted to assess the operator's ability to align the probe under different guidance modes. The network performance revealed the view of the imaging site and ultrasound images were transferred to the remote site in 233 ± 42 and 158 ± 38 ms, respectively. The expert sonographer was able to transfer, to the imaging site, data related to position and orientation of the ultrasound probe's hologram in 78 ± 13 ms. The user study indicated that the audiovisual cues are sufficient for an operator to position and orient a physical probe for accurate depiction of the targeted tissue (p < 0.001). The probe's placement translational and rotational errors were 1.4 ± 0.6 mm and 5.4 ± 2.2º. The work illustrates the feasibility of using a mixed-reality environment for effective communication between an expert sonographer (ultrasound physician) and an operator. Further studies are required to determine its applicability in a clinical setting during tele-ultrasound.

Pain Reliever for Parallelized People using Shortwave Diathermy

This project report details the development of a pain reliever for parallelized people using short wave diathermy. Short wave diathermy is an effective and safe method for pain relief in many conditions. It has the potential to reduce pain and improve the quality of life for individuals with chronic pain conditions. This project report outlines the development of a device that utilizes short wave diathermy to reduce pain in parallelized people. The device consists of a hand-held device that is placed over the affected area and a remote control. The device is designed to provide the user with the ability to adjust the intensity of the diathermy to their desired level. The device is also capable of tracking the user’s progress and providing feedback on the effectiveness of the therapy. Furthermore, the device is also able to provide the user with an audio-visual cue when the intensity of the therapy needs to be adjusted. The device is also capable of being used in conjunction with other pain relief methods, such as medications, physical therapy, and massage. The development of this device has the potential to provide relief to individuals with chronic pain conditions.