Audiovisual Training Research Articles

PO-0966: Audiovisual biofeedback breathing training during thoracic 4DCT imaging: a digital phantom study

PO-0966: Audiovisual biofeedback breathing training during thoracic 4DCT imaging: a digital phantom study

Cross-modal and multisensory training may distinctively shape restored senses.

Cross-modal and multisensory training may distinctively shape restored senses.

Plasticity of early neural language processes

Plasticity of early neural language processes

Audio-Visual Stimulation Improves Visual Search Abilities in Hemianopia due to Childhood Acquired Brain Lesions.

Results obtained in both animal models and hemianopic patients indicate that sound, spatially and temporally coincident with a visual stimulus, can improve visual perception in the blind hemifield, probably due to activation of 'multisensory neurons', mainly located in the superior colliculus. In view of this evidence, a new rehabilitation approach, based on audiovisual stimulation of visual field, has been proposed, and applied in adults with visual field reduction due to unilateral brain lesions. So far, results have been very encouraging, with improvements in visual search abilities. Based on these findings, we have investigated the possibility of inducing long-lasting amelioration also in children with a visual deficit due to acquired brain lesions. Our results suggest that, in the absence of spontaneous recovery, audiovisual training can induce activation of visual responsiveness of the oculomotor system also in children and adolescents with acquired lesions and confirm the putatively important role of the superior colliculus (SC) in this process.

Multisensory training can promote or impede visual perceptual learning of speech stimuli: visual-tactile vs. visual-auditory training

In a series of studies we have been investigating how multisensory training affects unisensory perceptual learning with speech stimuli. Previously, we reported that audiovisual (AV) training with speech stimuli can promote auditory-only (AO) perceptual learning in normal-hearing adults but can impede learning in congenitally deaf adults with late-acquired cochlear implants. Here, impeder and promoter effects were sought in normal-hearing adults who participated in lipreading training. In Experiment 1, visual-only (VO) training on paired associations between CVCVC nonsense word videos and nonsense pictures demonstrated that VO words could be learned to a high level of accuracy even by poor lipreaders. In Experiment 2, visual-auditory (VA) training in the same paradigm but with the addition of synchronous vocoded acoustic speech impeded VO learning of the stimuli in the paired-associates paradigm. In Experiment 3, the vocoded AO stimuli were shown to be less informative than the VO speech. Experiment 4 combined vibrotactile speech stimuli with the visual stimuli during training. Vibrotactile stimuli were shown to promote visual perceptual learning. In Experiment 5, no-training controls were used to show that training with visual speech carried over to consonant identification of untrained CVCVC stimuli but not to lipreading words in sentences. Across this and previous studies, multisensory training effects depended on the functional relationship between pathways engaged during training. Two principles are proposed to account for stimulus effects: (1) Stimuli presented to the trainee’s primary perceptual pathway will impede learning by a lower-rank pathway. (2) Stimuli presented to the trainee’s lower rank perceptual pathway will promote learning by a higher-rank pathway. The mechanisms supporting these principles are discussed in light of multisensory reverse hierarchy theory (RHT).

Development of an audiovisual speech perception app for children with autism spectrum disorders

Perception of spoken language requires attention to acoustic as well as visible phonetic information. This article reviews the known differences in audiovisual speech perception in children with autism spectrum disorders (ASD) and specifies the need for interventions that address this construct. Elements of an audiovisual training program are described. This researcher-developed program delivered via an iPad app presents natural speech in the context of increasing noise, but supported with a speaking face. Children are cued to attend to visible articulatory information to assist in perception of the spoken words. Data from four children with ASD ages 8–10 are presented showing that the children improved their performance on an untrained auditory speech-in-noise task.

Audiovisual spoken word training can promote or impede auditory-only perceptual learning: prelingually deafened adults with late-acquired cochlear implants versus normal hearing adults.

Training with audiovisual (AV) speech has been shown to promote auditory perceptual learning of vocoded acoustic speech by adults with normal hearing. In Experiment 1, we investigated whether AV speech promotes auditory-only (AO) perceptual learning in prelingually deafened adults with late-acquired cochlear implants. Participants were assigned to learn associations between spoken disyllabic C(=consonant)V(=vowel)CVC non-sense words and non-sense pictures (fribbles), under AV and then AO (AV-AO; or counter-balanced AO then AV, AO-AV, during Periods 1 then 2) training conditions. After training on each list of paired-associates (PA), testing was carried out AO. Across all training, AO PA test scores improved (7.2 percentage points) as did identification of consonants in new untrained CVCVC stimuli (3.5 percentage points). However, there was evidence that AV training impeded immediate AO perceptual learning: During Period-1, training scores across AV and AO conditions were not different, but AO test scores were dramatically lower in the AV-trained participants. During Period-2 AO training, the AV-AO participants obtained significantly higher AO test scores, demonstrating their ability to learn the auditory speech. Across both orders of training, whenever training was AV, AO test scores were significantly lower than training scores. Experiment 2 repeated the procedures with vocoded speech and 43 normal-hearing adults. Following AV training, their AO test scores were as high as or higher than following AO training. Also, their CVCVC identification scores patterned differently than those of the cochlear implant users. In Experiment 1, initial consonants were most accurate, and in Experiment 2, medial consonants were most accurate. We suggest that our results are consistent with a multisensory reverse hierarchy theory, which predicts that, whenever possible, perceivers carry out perceptual tasks immediately based on the experience and biases they bring to the task. We point out that while AV training could be an impediment to immediate unisensory perceptual learning in cochlear implant patients, it was also associated with higher scores during training.

Improving pulse oximetry pitch perception with multisensory perceptual training.

The pulse oximeter is a critical monitor in anesthesia practice designed to improve patient safety. Here, we present an approach to improve the ability of anesthesiologists to monitor arterial oxygen saturation via pulse oximetry through an audiovisual training process. Fifteen residents' abilities to detect auditory changes in pulse oximetry were measured before and after perceptual training. Training resulted in a 9% (95% confidence interval, 4%-14%, P = 0.0004, t(166) = 3.60) increase in detection accuracy, and a 72-millisecond (95% confidence interval, 40-103 milliseconds, P < 0.0001, t(166) = -4.52) speeding of response times in attentionally demanding and noisy conditions that were designed to simulate an operating room. This study illustrates the benefits of multisensory training and sets the stage for further work to better define the role of perceptual training in clinical anesthesiology.

Technical Efficiency in Shackled Actions after A Short Audiovisual Training Session

Introduction: Shackled maneuver was poorly studied in specific literature despite being a technique used every day for security and police forces. The use of new technologies had reported important benefits in learning process and could improve shackled learning process. Therefore, the objectives of the present research were: (i) to analyze the effect of an audiovisual training on the efficiency in shackled technique in three different stress situations and (ii) to study the week point in the shackled maneuver to know how to improve the audiovisual training. Methods: Technical procedures in shackle techniques in 3 different situations of 26 male soldiers were analyzed after an audiovisual session. The situations were normal, alert and danger. All the shackle interventions were filmed by a video camera and were analyzed later. Shackle maneuver was divided in approach to the subject, subject control, placing the first shackle, limb immobilization, placing the second shackle, frisk and transfer. Every part of the shackle maneuver was evaluated with 10, 5 or 0 point. Results: Maneuver efficiency decreased with the stress. There was significant difference in the placement of the second phase shackle between the normal and alert situations and in all the situations the limb immobilization phase obtained the lower values. Furthermore, the execution time was lower in normal (40.30 s) than in alert (120.08 s) and danger situations (180.36 s). Conclusion: A short audiovisual training was enough to learn the proper shackled procedure in non experimented soldiers with a low technical level in the limb immobilization phase and differences between the stress situation in the placing the second shackle phase. Also, the increase in stress caused an increase in the maneuver time, and an increase in number of the video reproductions.

Neural plasticity in phonetic training of the /i-I/ contrast for adult Chinese speakers

This study investigated neural plasticity associated with phonetic training using a software program developed after Zhang et al. [NeuroImage 46, 226–240 (2009)]. The target sounds were /i/ and /I/ in English, a non-phonemic contrast in Mandarin Chinese. The training program integrated four levels of spectro-temporal exaggerations, multi-talker variability, audio-visual presentation, and adaptive listening in seven sessions, each lasting about 15 min. The participants were ten adult Chinese English-as-a-second-language learners. Identical pre- and post-tests were administered one week before and after training. Behavioral measures included discrimination and identification tasks as well as formant analysis of vowel production. Event Related Potential (ERP) measures examined training-induced changes in the mismatch negativity (MMN) responses. The behavioral results showed significant improvement in identification and discrimination scores and a clear continuous-to-categorical perceptual shift, which were also reflected in the MMN responses for detecting the across- vs within-category differences at the pre-attentive level. There was also strong evidence for transfer of learning from trained to untrained stimuli as well as from perception to production. The results demonstrate the existence of substantial neural plasticity for speech learning in adulthood and provide further testimony for the efficacy of the adaptive audiovisual training method for promoting second language phonetic learning.

Effect of long-term training on sound localization performance with spectrally warped and band-limited head-related transfer functions

Sound localization in the sagittal planes, including the ability to distinguish front from back, relies on spectral features caused by the filtering effects of the head, pinna, and torso. It is assumed that important spatial cues are encoded in the frequency range between 4 and 16 kHz. In this study, in a double-blind design and using audio-visual training covering the full 3-D space, normal-hearing listeners were trained 2 h per day over three weeks to localize sounds which were either band limited up to 8.5 kHz or spectrally warped from the range between 2.8 and 16 kHz to the range between 2.8 and 8.5 kHz. The training effect for the warped condition exceeded that for procedural task learning, suggesting a stable auditory recalibration due to the training. After the training, performance with band-limited sounds was better than that with warped ones. The results show that training can improve sound localization in cases where spectral cues have been reduced by band-limiting or remapped by warping. This suggests that hearing-impaired listeners, who have limited access to high frequencies, might also improve their localization ability when provided with spectrally warped or band-limited sounds and adequately trained on sound localization.

A comprehensive system for facial animation of generic 3D head models driven by speech

A comprehensive system for facial animation of generic 3D head models driven by speech is presented in this article. In the training stage, audio-visual information is extracted from audio-visual training data, and then used to compute the parameters of a single joint audio-visual hidden Markov model (AV-HMM). In contrast to most of the methods in the literature, the proposed approach does not require segmentation/classification processing stages of the audio-visual data, avoiding the error propagation related to these procedures. The trained AV-HMM provides a compact representation of the audio-visual data, without the need of phoneme (word) segmentation, which makes it adaptable to different languages. Visual features are estimated from the speech signal based on the inversion of the AV-HMM. The estimated visual speech features are used to animate a simple face model. The animation of a more complex head model is then obtained by automatically mapping the deformation of the simple model to it, using a small number of control points for the interpolation. The proposed algorithm allows the animation of 3D head models of arbitrary complexity through a simple setup procedure. The resulting animation is evaluated in terms of intelligibility of visual speech through perceptual tests, showing a promising performance. The computational complexity of the proposed system is analyzed, showing the feasibility of its real-time implementation.

Early gamma oscillations during rapid auditory processing in children with a language-learning impairment: Changes in neural mass activity after training

Children with language-learning impairment (LLI) have consistently shown difficulty with tasks requiring precise, rapid auditory processing. Remediation based on neural plasticity assumes that the temporal precision of neural coding can be improved by intensive training protocols. Here, we examined the extent to which early oscillatory responses in auditory cortex change after audio-visual training, using combined source modeling and time-frequency analysis of the human electroencephalogram (EEG). Twenty-one elementary school students diagnosed with LLI underwent the intervention for an average of 32 days. Pre- and post-training assessments included standardized language/literacy tests and EEG recordings in response to fast-rate tone doublets. Twelve children with typical language development were also tested twice, with no intervention given. Behaviorally, improvements on measures of language were observed in the LLI group following completion of training. During the first EEG assessment, we found reduced amplitude and phase-locking of early (45–75ms) oscillations in the gamma-band range (29–52Hz), specifically in the LLI group, for the second stimulus of the tone doublet. Amplitude reduction for the second tone was no longer evident for the LLI children post-intervention, although these children still exhibited attenuated phase-locking. Our findings suggest that specific aspects of inefficient sensory cortical processing in LLI are ameliorated after training.

Auditory Perceptual Learning for Speech Perception Can be Enhanced by Audiovisual Training

Speech perception under audiovisual (AV) conditions is well known to confer benefits to perception such as increased speed and accuracy. Here, we investigated how AV training might benefit or impede auditory perceptual learning of speech degraded by vocoding. In Experiments 1 and 3, participants learned paired associations between vocoded spoken nonsense words and nonsense pictures. In Experiment 1, paired-associates (PA) AV training of one group of participants was compared with audio-only (AO) training of another group. When tested under AO conditions, the AV-trained group was significantly more accurate than the AO-trained group. In addition, pre- and post-training AO forced-choice consonant identification with untrained nonsense words showed that AV-trained participants had learned significantly more than AO participants. The pattern of results pointed to their having learned at the level of the auditory phonetic features of the vocoded stimuli. Experiment 2, a no-training control with testing and re-testing on the AO consonant identification, showed that the controls were as accurate as the AO-trained participants in Experiment 1 but less accurate than the AV-trained participants. In Experiment 3, PA training alternated AV and AO conditions on a list-by-list basis within participants, and training was to criterion (92% correct). PA training with AO stimuli was reliably more effective than training with AV stimuli. We explain these discrepant results in terms of the so-called “reverse hierarchy theory” of perceptual learning and in terms of the diverse multisensory and unisensory processing resources available to speech perception. We propose that early AV speech integration can potentially impede auditory perceptual learning; but visual top-down access to relevant auditory features can promote auditory perceptual learning.

A Identidade Docente Face às Novas Tecnologias

With the imminent arrival of Digital TV in the Brazilian schools, this research investigated the perceptions of teachers about the possibility of producing educational materials for that media inside the school. There were 50 participants among regent teachers, teachers who have already been producing educational materials and future pedagogues that participated in the survey through interviews and questionnaires. The results show an average of 80% who realize digital television is as a vast repository of contents. Although they have skills to use different media, technological devices, tools for communication and social interaction, the possibility of producing audio-visual educational content is not yet clearly considered as important by most of them. From the data, it was possible to reach two conditioning factors that can contribute for teachers´ audiovisual training.

Evidence for Training-Induced Plasticity in Multisensory Brain Structures: An MEG Study

Multisensory learning and resulting neural brain plasticity have recently become a topic of renewed interest in human cognitive neuroscience. Music notation reading is an ideal stimulus to study multisensory learning, as it allows studying the integration of visual, auditory and sensorimotor information processing. The present study aimed at answering whether multisensory learning alters uni-sensory structures, interconnections of uni-sensory structures or specific multisensory areas. In a short-term piano training procedure musically naive subjects were trained to play tone sequences from visually presented patterns in a music notation-like system [Auditory-Visual-Somatosensory group (AVS)], while another group received audio-visual training only that involved viewing the patterns and attentively listening to the recordings of the AVS training sessions [Auditory-Visual group (AV)]. Training-related changes in cortical networks were assessed by pre- and post-training magnetoencephalographic (MEG) recordings of an auditory, a visual and an integrated audio-visual mismatch negativity (MMN). The two groups (AVS and AV) were differently affected by the training. The results suggest that multisensory training alters the function of multisensory structures, and not the uni-sensory ones along with their interconnections, and thus provide an answer to an important question presented by cognitive models of multisensory training.

Visual benefit in bimodal training with highly distorted speech sound

Artificial auditory devices such as cochlear implants (CIs) and auditory brainstem implants (ABIs) have become standard means to manage profound sensorineural hearing loss. However, because of their structural limitations compared to the cochlea and the cochlear nucleus, the generated auditory sensations are still imperfect. Recipients need postoperative auditory rehabilitation. To improve these rehabilitation programs, this study evaluated the effects of bimodal (audio–visual) training under seven experimental conditions of distorted speech sound, named noise-vocoded speech sound (NVSS), which is similarly processed with a speech processor of CI/ABI. Word intelligibilities under the seven conditions of two-band noise-vocoded speech were measured for auditory (A), visual (V) and auditory–visual (AV) modalities after a few hours of bimodal (AV) training. The experiment was performed with 56 subjects with normal hearing. Performance of A and AV word recognition was significantly different under the seven auditory conditions. The V word intelligibility was not influenced by the condition of combined auditory cues. However, V word intelligibility was correlated with AV word recognition under all frequency conditions. Correlation between A and AV word intelligibilities was ambiguous. These findings suggest the importance of visual cues in AV speech perception under extremely degraded auditory conditions, and underscore the importance of the possible effectiveness of bimodal audio–visual training in postoperative rehabilitation for patients with postlingual deafness who have undergone artificial auditory device implantation.

Mapping Symbols to Sounds: Electrophysiological Correlates of the Impaired Reading Process in Dyslexia

Dyslexic and control first-grade school children were compared in a Symbol-to-Sound matching test based on a non-linguistic audiovisual training which is known to have a remediating effect on dyslexia. Visual symbol patterns had to be matched with predicted sound patterns. Sounds incongruent with the corresponding visual symbol (thus not matching the prediction) elicited the N2b and P3a event-related potential (ERP) components relative to congruent sounds in control children. Their ERPs resembled the ERP effects previously reported for healthy adults with this paradigm. In dyslexic children, N2b onset latency was delayed and its amplitude significantly reduced over left hemisphere whereas P3a was absent. Moreover, N2b amplitudes significantly correlated with the reading skills. ERPs to sound changes in a control condition were unaffected. In addition, correctly predicted sounds, that is, sounds that are congruent with the visual symbol, elicited an early induced auditory gamma band response (GBR) reflecting synchronization of brain activity in normal-reading children as previously observed in healthy adults. However, dyslexic children showed no GBR. This indicates that visual symbolic and auditory sensory information are not integrated into a unitary audiovisual object representation in them. Finally, incongruent sounds were followed by a later desynchronization of brain activity in the gamma band in both groups. This desynchronization was significantly larger in dyslexic children. Although both groups accomplished the task successfully remarkable group differences in brain responses suggest that normal-reading children and dyslexic children recruit (partly) different brain mechanisms when solving the task. We propose that abnormal ERPs and GBRs in dyslexic readers indicate a deficit resulting in a widespread impairment in processing and integrating auditory and visual information and contributing to the reading impairment in dyslexia.

Effect of comprehensive nursing intervention on the treatment effect of common bile duct stones in patients undergoing endoscopy

Objective To investigate the effect of comprehensive nursing intervention on the treatment of common bile duct stones undergoing endoscopy. Methods A total of 137 patients with common bile duct stones undergoing the endoscopic treatment were randomly divided into intervention group （ n = 69 cases） and control group （ n = 68 patients）. All the patients received routine care before and after the endoscopic treatment. In addition, the patients in the intervention group received audio-visual training intervention with preoperative, music therapy with intra-operative and position training intervention with postoperative. The intervention effect was compared between the two groups. Results Breath rate, heart rate, oxygen saturation before treatment were not different in the two groups （ P 〉 0. 05 ）. Breath frequency in intervention group after treatment was （ 23.6±3.4）time/rain, the first bed time was （19.24±3.67）h, extubation rate was 97. 10% and complication rate was 4.35% and those in control group were （28.9±3.7） time/min, （24.62±4.81 ） h, 86.76% and 16.18% respectively, and the differences were significant （ t = 8.72, 7.35 ; X^2 = 4.96, 5.03 ; P 〈 0. 05 ）. Heart rate, blood pressure in experimental group after treatment were reduced and the extubation time and in hospital time were shorten than control group, however, the sleeping time was longer in experimental group （ P 〈 0. 05 ）. Conclusions Comprehensive nursing intervention can improve treatment compliance, promote patients＇ early recovery and achieve the desired effect. Key words: Common bile duct stones; Endoscopy therapy; Comprehensive nursing intervention

AP067 ERC BLS AED course vs. audio-visual training with prompt feedback – Pilot study

s, Resuscitation 2011 – Implementation / Resuscitation 82S1 (2011) S1–S34 S25 with moderate to good cerebral performance. So, research hypothesis was confirmed (p<0.05). Discussion and Conclusion: “CPCR management process” might be applied as a model for increasing resuscitation outcomes. In patients under CPCR by health care managers in hospitals.