Auditory-visual Research Articles

Variable effects of speakers’ visual cues and accent on L2 listening comprehension: A mixed-methods approach

Few studies have explored the influence of a speaker’s accent and visual (facial and gestural) cues on second-language (L2) listening comprehension. The current mixed-methods between-groups design investigated: (1) the effects of accent and visual cues on Arab students’ comprehension of recorded lectures delivered by two speakers: first language (L1) American English and second language (L2) English (strong Vietnamese accent), (2) students’ assessments of the comprehensibility and English nativelikeness of the accents, and (3) students’ perceptions of and preferences for visual cues in communication. A total of 120 Arab university students were distributed across three stimulus conditions: auditory-visual (AV)-gesture-face, AV-face, and A-only within two accent conditions (L1, L2 English). Materials included: a listening comprehension task, 9-point rating scales for comprehensibility and nativelikeness of accent, and a speaker’s accent and visual cues (SAVC) questionnaire. ANCOVAs were conducted on listening comprehension scores and the ratings, with students’ listening proficiency as the covariate. Students with higher listening proficiency received higher comprehension scores and tended to recognize the L1 English accent as nativelike; however, proficiency was not significantly related to their comprehensibility ratings of either accent. The L1 English speaker’s gestures were associated with higher comprehensibility ratings; the L2 English speaker’s gestures were associated with a stronger nonnativelike accent, which students found frustrating to comprehend. Although SAVC responses revealed a general preference for facial cues and gestures in daily communication, the L2 English speaker’s gestures in the study were not considered helpful. Thematic analysis of interviews emphasized: (1) the variable effectiveness of visual cues, (2) the important role of topic and accent familiarity, (3) the relationship between listening proficiency and interlocutor/instructor preference, and (4) strategies to deal with unfamiliar accents. Pedagogical implications include increased awareness of variability in visual cues across speakers and the need to build familiarity with various accents and gesture patterns.

Effects of simulated multi-sensory stimulation integration on physiological and psychological restoration in virtual urban green space environment.

Virtual urban green environment images and audio stimuli had been proven to have restorative effects on subjects' physical and mental health. In this area, researchers predominantly focused on visual, auditory and olfactory aspects, while tactile and gustatory senses have been minimally explored. However, the optimal combination of sensory stimuli for promoting physical and mental recovery remains unclear. Therefore, a simulated sensory stimulation approach involving 240 participants was employed, with 30 individuals included in each of the eight experimental groups: the visual-auditory (VA), visual-auditory-olfactory (VAO), visual-auditory-tactile (VAT), visual-auditory-gustatory(VAG), visual-auditory-olfactory-tactile (VAOT), visual-auditory-olfactory-gustatory (VAOG), visual-auditory-tactile-gustatory (VATG), and visual-auditory-olfactory-tactile-gustatory (VAOTG) groups. This study aimed to explore the differences in participants' physiological and psychological health recovery after exposure to different combinations of simulated sensory stimuli in virtual UGSs. The results indicated that the following: (1) In terms of physiological recovery, the blood pressure of the 8 experimental groups decreased significantly after the experiment, indicating that the virtual urban green space environment has a certain recovery effect on physiological state. The combination of VAOTG stimuli in the multisensory group resulted in the best blood pressure recovery (p < 0.05). Tactile is an important sense to enhance the physiological recovery effect. Olfactory-tactile or tactile-gustatory stimuli interactions significantly enhance physiological recovery, emphasizing the importance of tactile stimulation in improving physiological recovery. (2) In terms of psychological recovery, the common trigger of olfactory-gustatory is the most key element to enhance psychological recovery through multi-sensory stimulation of virtual urban green space environment. VAOG stimulation had the best effect on psychological recovery (p < 0.05), followed by VAOTG stimulation (p < 0.05). Gustatory is an important sense to enhance the psychological recovery effect, and both the tactile-gustatory interaction and the olfactory-gustatory interaction significantly enhance the recovery effect. At the same time, the psychological recovery effect obtained by four or more sensory combinations was higher than that obtained by two or three sensory stimulation groups. This study confirms more possibilities for ways to restore physical and mental health through virtual natural environments. It expands the research on the benefits of virtual nature experience and provides theoretical support for the application of this method.

Seeing a Talking Face Matters: Gaze Behavior and the Auditory-Visual Speech Benefit in Adults' Cortical Tracking of Infant-directed Speech.

In face-to-face conversations, listeners gather visual speech information from a speaker's talking face that enhances their perception of the incoming auditory speech signal. This auditory-visual (AV) speech benefit is evident even in quiet environments but is stronger in situations that require greater listening effort such as when the speech signal itself deviates from listeners' expectations. One example is infant-directed speech (IDS) presented to adults. IDS has exaggerated acoustic properties that are easily discriminable from adult-directed speech (ADS). Although IDS is a speech register that adults typically use with infants, no previous neurophysiological study has directly examined whether adult listeners process IDS differently from ADS. To address this, the current study simultaneously recorded EEG and eye-tracking data from adult participants as they were presented with auditory-only (AO), visual-only, and AV recordings of IDS and ADS. Eye-tracking data were recorded because looking behavior to the speaker's eyes and mouth modulates the extent of AV speech benefit experienced. Analyses of cortical tracking accuracy revealed that cortical tracking of the speech envelope was significant in AO and AV modalities for IDS and ADS. However, the AV speech benefit [i.e., AV > (A + V)] was only present for IDS trials. Gaze behavior analyses indicated differences in looking behavior during IDS and ADS trials. Surprisingly, looking behavior to the speaker's eyes and mouth was not correlated with cortical tracking accuracy. Additional exploratory analyses indicated that attention to the whole display was negatively correlated with cortical tracking accuracy of AO and visual-only trials in IDS. Our results underscore the nuances involved in the relationship between neurophysiological AV speech benefit and looking behavior.

Holographic Hintways: A systems feasibility and usability study of augmented reality cueing for gait adaptation

BackgroundThrough providing on-demand visual and auditory cues while walking,augmented reality (AR) can theoretically cue spatiotemporal gait adaptations in, populations such as those with Parkinson’s disease. However, given the novelty of the, technology, the type and extent of gait adaptations in response to such a cueing, system are unknown. Before such systems can be incorporated into rehabilitation, approaches, it is important to understand how people interact with the technology. Research questions: What are the effects of visual and auditory cues delivered, through AR on spatiotemporal walking patterns and variability in a healthy, young, population? Is there a relationship between system usability and gait variability?, MethodsTwenty healthy, young participants walked in four different cueing conditions using an AR headset: No Cues (NC) (i.e., natural gait), Auditory (A), Visual (V), and Auditory + Visual (AV). Inertial measurement units recorded spatiotemporal gait data at 200 Hz, a System Usability Survey was administered afterward, and linear regression models were developed to examine whether gait variability is predictive of system usability. ResultsAll cueing conditions exhibited a significantly slower cadence compared to, NC trials. Cadence variability was significantly higher for A trials compared to V and, NC. V trials exhibited significantly decreased stride lengths compared to NC. Increased, reported system usability was significantly correlated with decreased stance phase, time variability across A trials. SignificanceOur findings support that holographic spatial-visual and auditory cues, are promising to evoke spatiotemporal gait adaptations. Results also support the, notion that the type of system and cue delivery design may impact gait outcomes.,Before an AR cueing system can be applied to a specific population in future, interventions, a more holistic approach towards finding the relationship between this, technology and its users is needed.

Multi-modal cross-linguistic perception of Mandarin tones in clear speech.

Clearly enunciated speech (relative to conversational, plain speech) involves articulatory and acoustic modifications that enhance auditory-visual (AV) segmental intelligibility. However, little research has explored clear-speech effects on the perception of suprasegmental properties such as lexical tone, particularly involving visual (facial) perception. Since tone production does not primarily rely on vocal tract configurations, tones may be less visually distinctive. Questions thus arise as to whether clear speech can enhance visual tone intelligibility, and if so, whether any intelligibility gain can be attributable to tone-specific category-enhancing (code-based) clear-speech cues or tone-general saliency-enhancing (signal-based) cues. The present study addresses these questions by examining the identification of clear and plain Mandarin tones with visual-only, auditory-only, and AV input modalities by native (Mandarin) and nonnative (English) perceivers. Results show that code-based visual and acoustic clear tone modifications, although limited, affect both native and nonnative intelligibility, with category-enhancing cues increasing intelligibility and category-blurring cues decreasing intelligibility. In contrast, signal-based cues, which are extensively available, do not benefit native intelligibility, although they contribute to nonnative intelligibility gain. These findings demonstrate that linguistically relevant visual tonal cues are existent. In clear speech, such tone category-enhancing cues are incorporated with saliency-enhancing cues across AV modalities for intelligibility improvements.

Older adults' emotion recognition: No auditory-visual benefit for less clear expressions.

The ability to recognise emotion from faces or voices appears to decline with advancing age. However, some studies have shown that emotion recognition of auditory-visual (AV) expressions is largely unaffected by age, i.e., older adults get a larger benefit from AV presentation than younger adults resulting in similar AV recognition levels. An issue with these studies is that they used well-recognised emotional expressions that are unlikely to generalise to real-life settings. To examine if an AV emotion recognition benefit generalizes across well and less well recognised stimuli, we conducted an emotion recognition study using expressions that had clear or unclear emotion information for both modalities, or clear visual, but unclear auditory information. Older (n = 30) and younger (n = 30) participants were tested on stimuli of anger, happiness, sadness, surprise, and disgust (expressed in spoken sentences) in auditory-only (AO), visual-only (VO), or AV format. Participants were required to respond by choosing one of 5 emotion options. Younger adults were more accurate in recognising emotions than older adults except for clear VO expressions. Younger adults showed an AV benefit even when unimodal recognition was poor. No such AV benefit was found for older adults; indeed, AV was worse than VO recognition when AO recognition was poor. Analyses of confusion responses indicated that older adults generated more confusion responses that were common between AO and VO conditions, than younger adults. We propose that older adults' poorer AV performance may be due to a combination of weak auditory emotion recognition and response uncertainty that resulted in a higher cognitive load.

Seeing a talking face matters: Infants' segmentation of continuous auditory-visual speech.

Visual speech cues from a speaker's talking face aid speech segmentation in adults, but despite the importance of speech segmentation in language acquisition, little is known about the possible influence of visual speech on infants' speech segmentation. Here, to investigate whether there is facilitation of speech segmentation by visual information, two groups of English-learning 7-month-old infants were presented with continuous speech passages, one group with auditory-only (AO) speech and the other with auditory-visual (AV) speech. Additionally, the possible relation between infants' relative attention to the speaker's mouth versus eye regions and their segmentation performance was examined. Both the AO and the AV groups of infants successfully segmented words from the continuous speech stream, but segmentation performance persisted for longer for infants in the AV group. Interestingly, while AV group infants showed no significant relation between the relative amount of time spent fixating the speaker's mouth versus eyes and word segmentation, their attention to the mouth was greater than that of AO group infants, especially early in test trials. The results are discussed in relation to the possible pathways through which visual speech cues aid speech perception.

Listeners’ attitudes towards voice disorders: An interaction between auditory and visual stimuli

ObjectivePeople with dysphonia are judged more negatively than peers with normal vocal quality. This preliminary study aims to (1) investigate correlations between both auditory-perceptual and objective measures of vocal quality of dysphonic and non-dysphonic speakers and attitudes of listeners, and (2) discover whether these attitudes towards people with dysphonia vary for different types of stimuli: auditory (A) stimuli and combined auditory-visual (AV) stimuli. Visual (V) stimuli were included as a control condition. MethodTen judges with no experience in the evaluation of dysphonia were asked to rate A, AV and V stimuli of 14 different speakers (10 dysphonic and 4 non-dysphonic speakers) Cognitive attitudes, evaluation of voice characteristics and behavioral attitudes were examined. Pearson and Spearman correlation coefficients were calculated to examine correlations between both Dysphonia Severity Index (DSI) values and perceptual vocal quality as assessed by a speech-language pathologist (PVQSLP) or perceptual vocal quality as assessed by the judges (PVQjudge). Linear mixed model (LMM) analyses were conducted to investigate differences between speakers and stimuli conditions. ResultsStatistically significant correlations were found between both perceptual and objective measures of vocal quality and mean attitude scores for A and AV stimuli, indicating increasingly negative attitudes with increasing dysphonia severity. Fewer statistically significant correlations were found for the combined AV stimuli than for A stimuli, and no significant correlations were found for V stimuli. LMM analyses revealed significant group effects for several cognitive attitudes. ConclusionGenerally, people with dysphonia are judged more negatively by listeners than peers without dysphonia. However, the findings of this study suggest a positive influence of visual cues on the judges’ cognitive and behavioral attitudes towards dysphonic speakers. Further research is needed to investigate the significance of this influence.

Seeing a talking face matters: The relationship between cortical tracking of continuous auditory‐visual speech and gaze behaviour in infants, children and adults

An auditory-visual speech benefit, the benefit that visual speech cues bring to auditory speech perception, is experienced from early on in infancy and continues to be experienced to an increasing degree with age. While there is both behavioural and neurophysiological evidence for children and adults, only behavioural evidence exists for infants – as no neurophysiological study has provided a comprehensive examination of the auditory-visual speech benefit in infants. It is also surprising that most studies on auditory-visual speech benefit do not concurrently report looking behaviour especially since the auditory-visual speech benefit rests on the assumption that listeners attend to a speaker's talking face and that there are meaningful individual differences in looking behaviour. To address these gaps, we simultaneously recorded electroencephalographic (EEG) and eye-tracking data of 5-month-olds, 4-year-olds and adults as they were presented with a speaker in auditory-only (AO), visual-only (VO), and auditory-visual (AV) modes. Cortical tracking analyses that involved forward encoding models of the speech envelope revealed that there was an auditory-visual speech benefit [i.e., AV > (A + V)], evident in 5-month-olds and adults but not 4-year-olds. Examination of cortical tracking accuracy in relation to looking behaviour, showed that infants’ relative attention to the speaker's mouth (vs. eyes) was positively correlated with cortical tracking accuracy of VO speech, whereas adults’ attention to the display overall was negatively correlated with cortical tracking accuracy of VO speech. This study provides the first neurophysiological evidence of auditory-visual speech benefit in infants and our results suggest ways in which current models of speech processing can be fine-tuned.

Seeing lexical tone: Head and face motion in production and perception of Cantonese lexical tones

Previous studies show that lexical tones can be discriminated visually, but the locus of this information is unknown. Here we investigate the role of visual face and head information in the production and perception of the six Cantonese lexical tones. Experiment 1 (speech production) showed that tone category membership is predicted independently by each of three visual components, Head-only, Face-only, and combined Face&Head motion, that Face&Head combined is a better predictor than the sum of Head-only and Face-only, that Head-only was a better predictor than Face-only, and that up-down head motion may be involved. In Experiment 2 (speech perception) discrimination of Cantonese tones and phones (consonants and vowels) was investigated in Auditory-Only (AO), Visual-Only (VO), and Auditory-Visual (AV) presentations of Face-only, Head-only or combined Face&Head information. There was successful visual discrimination of both phones and tones, but in different manners. For VO, face movement (Face-only, Face&Head) was sufficient for Phone discrimination, whereas both head and face (Face&Head) were necessary for Tone discrimination; and Phone discrimination was better for Face-only than Head-only, whereas Tone discrimination was the better for Head-only than Face-only. For visual augmentation of auditory information, (AV-AO)/AO), for Phones there was significant facilitation in Head-only and Face-only, but not Face&Head, whereas for Tones there was significant facilitation only in the Face&Head condition. Overall, head motion is integrally involved in the differential production of Cantonese tones and, as for consonants and vowels, there is visual perception of lexical tone, albeit based on entirely different cues: face motion is sufficient for phone perception (head motion is not necessary), whereas head motion is necessary but not sufficient for tone perception (Face&Head required). Head motion is integral both to the differential production of Cantonese tones and to the visual discrimination of those tones.

Detection and Recognition of Asynchronous Auditory/Visual Speech: Effects of Age, Hearing Loss, and Talker Accent.

This investigation examined age-related differences in auditory-visual (AV) integration as reflected on perceptual judgments of temporally misaligned AV English sentences spoken by native English and native Spanish talkers. In the detection task, it was expected that slowed auditory temporal processing of older participants, relative to younger participants, would be manifest as a shift in the range over which participants would judge asynchronous stimuli as synchronous (referred to as the “AV simultaneity window”). The older participants were also expected to exhibit greater declines in speech recognition for asynchronous AV stimuli than younger participants. Talker accent was hypothesized to influence listener performance, with older listeners exhibiting a greater narrowing of the AV simultaneity window and much poorer recognition of asynchronous AV foreign-accented speech compared to younger listeners. Participant groups included younger and older participants with normal hearing and older participants with hearing loss. Stimuli were video recordings of sentences produced by native English and native Spanish talkers. The video recordings were altered in 50 ms steps by delaying either the audio or video onset. Participants performed a detection task in which they judged whether the sentences were synchronous or asynchronous, and performed a recognition task for multiple synchronous and asynchronous conditions. Both the detection and recognition tasks were conducted at the individualized signal-to-noise ratio (SNR) corresponding to approximately 70% correct speech recognition performance for synchronous AV sentences. Older listeners with and without hearing loss generally showed wider AV simultaneity windows than younger listeners, possibly reflecting slowed auditory temporal processing in auditory lead conditions and reduced sensitivity to asynchrony in auditory lag conditions. However, older and younger listeners were affected similarly by misalignment of auditory and visual signal onsets on the speech recognition task. This suggests that older listeners are negatively impacted by temporal misalignments for speech recognition, even when they do not notice that the stimuli are asynchronous. Overall, the findings show that when listener performance is equated for simultaneous AV speech signals, age effects are apparent in detection judgments but not in recognition of asynchronous speech.

Effects of Age and Uncertainty on the Visual Speech Benefit in Noise.

Listeners understand significantly more speech in noise when the talker's face can be seen (visual speech) in comparison to an auditory-only baseline (a visual speech benefit). This study investigated whether the visual speech benefit is reduced when the correspondence between auditory and visual speech is uncertain and whether any reduction is affected by listener age (older vs. younger) and how severe the auditory signal is masked. Older and younger adults completed a speech recognition in noise task that included an auditory-only condition and four auditory-visual (AV) conditions in which one, two, four, or six silent talking face videos were presented. One face always matched the auditory signal; the other face(s) did not. Auditory speech was presented in noise at -6 and -1 dB signal-to-noise ratio (SNR). When the SNR was -6 dB, for both age groups, the standard-sized visual speech benefit reduced as more talking faces were presented. When the SNR was -1 dB, younger adults received the standard-sized visual speech benefit even when two talking faces were presented, whereas older adults did not. The size of the visual speech benefit obtained by older adults was always smaller when AV correspondence was uncertain; this was not the case for younger adults. Difficulty establishing AV correspondence may be a factor that limits older adults' speech recognition in noisy AV environments. Supplemental Material https://doi.org/10.23641/asha.16879549.

MEDIA PEMBELAJARAN AUDIO VISUAL: LITERATURE REVIEW

The use of media and learning methods is the right solution for basic education teachers to develop students' skills. The media most likely to be applied in schools is the use of audio media. The research objective was to determine the benefits of audio-visual media in learning the speaking skills of low-grade students (Elementary School). This type of research uses qualitative research with literature review methods. The library selection technique is based on audiovisual media keywords on speaking skills. The literature comes from accredited national journals on Google Scholar with a publication limit of the last 10 years. The literature study technique uses a synthetic matrix based on reference sources, types of samples, methods, interventions, and findings. The results obtained in this study are as many as 12 journals about audio-visual learning media. The conclusion of this study is that it is found that audio-visual media that can be applied in the learning process include the audiobilingual method assisted by wayang media, the VAKS method (Visual, auditors, kinesthetic, suggestopedia), role playing learning using audio-visual media, the re-reading method - said with the aid of a tape recorder, learning model SAVI (Somatis Auditory Visual Intellectual), learning using film media, learning retelling methods using audio-visual media.

The neural bases of multimodal sensory integration in older adults

Although hearing often declines with age, prior research has shown that older adults may benefit from multisensory input to a greater extent when compared to younger adults, a concept known as inverse effectiveness. While there is behavioral evidence in support of this phenomenon, less is known about its neural basis. The present functional MRI (fMRI) study examined how older and younger adults processed multimodal auditory-visual (AV) phonemic stimuli which were either congruent or incongruent across modalities. Incongruent AV pairs were designed to elicit the McGurk effect. Behaviorally, reaction times were significantly faster during congruent trials compared to incongruent trials for both age-groups, and overall older adults responded more slowly. The interaction was not significant, suggesting that older adults processed the AV stimuli similarly to younger adults. Although there were minimal behavioral differences, age-related differences in functional activation were identified: Younger adults elicited greater activation than older adults in primary sensory regions including superior temporal gyrus, the calcarine fissure, and left postcentral gyrus. In contrast, older adults elicited greater activation than younger adults in dorsal frontal regions including middle and superior frontal gyri, as well as dorsal parietal regions. These data suggest that while there is age-related stability in behavioral sensitivity to multimodal stimuli, the neural bases for this effect differed between older and younger adults. Our results demonstrated that older adults underrecruited primary sensory cortices and had increased recruitment of regions involved in executive function, attention, and monitoring processes, which may reflect an attempt to compensate.

Development of Ludo-Science Media with a Somatic Auditory Visual Intellectual (SAVI) Approach to Train the Activeness and Conceptual Understanding

This study is aimed to develop an educational game media called ludo-science that is compiled with Somatic Auditory Visualization Intellectually (SAVI) approach which can be used to train the activeness and students' conceptual understanding. The development procedure is reference to the ADDIE development model. The research data were obtained in the form of validation data, readability data, response data, activeness data, and pretest posttest scores. The results showed that after going through the product validation process by material experts obtained an average percentage of 92,70% and included in the category of very feasible. Product validation by media experts obtained an average percentage of 82,39% and included in the very feasible category. The average readability gained 95,75% and included in the excellent category. The results of activeness obtained an average activity of students by 82,94% and included in the category of very active. The pretest and posttest showed an increase in students' conceptual understanding. The classical calculation state that at the pretest the concept understands is 37,27% while at the posttest is 63,59%. The average questionnaire responses for the use of Ludo-science media with SAVI received an average of 86,79% and included in the excellent category.

Multimodal input in second-language speech processing

This timeline provides an update on research since 2009 involving auditory-visual (AV) input in spoken language processing (see Hardison, 2010 for an earlier timeline on this topic). A brief background is presented here as a foundation for the more recent studies of speech as a multimodal phenomenon (e.g., Rosenblum, 2005).

Modality Effects on Lexical Encoding and Memory Representations of Spoken Words.

The present study investigated presentation modality differences in lexical encoding and working memory representations of spoken words of older, hearing-impaired adults. Two experiments were undertaken: a memory-scanning experiment and a stimulus gating experiment. The primary objective of experiment 1 was to determine whether memory encoding and retrieval and scanning speeds are different for easily identifiable words presented in auditory-visual (AV), auditory-only (AO), and visual-only (VO) modalities. The primary objective of experiment 2 was to determine if memory encoding and retrieval speed differences observed in experiment 1 could be attributed to the early availability of AV speech information compared with AO or VO conditions. Twenty-six adults over age 60 years with bilateral mild to moderate sensorineural hearing loss participated in experiment 1, and 24 adults who took part in experiment 1 participated in experiment 2. An item recognition reaction-time paradigm (memory-scanning) was used in experiment 1 to measure (1) lexical encoding speed, that is, the speed at which an easily identifiable word was recognized and placed into working memory, and (2) retrieval speed, that is, the speed at which words were retrieved from memory and compared with similarly encoded words (memory scanning) presented in AV, AO, and VO modalities. Experiment 2 used a time-gated word identification task to test whether the time course of stimulus information available to participants predicted the modality-related memory encoding and retrieval speed results from experiment 1. The results of experiment 1 revealed significant differences among the modalities with respect to both memory encoding and retrieval speed, with AV fastest and VO slowest. These differences motivated an examination of the time course of stimulus information available as a function of modality. Results from experiment 2 indicated the encoding and retrieval speed advantages for AV and AO words compared with VO words were mostly driven by the time course of stimulus information. The AV advantage seen in encoding and retrieval speeds is likely due to a combination of robust stimulus information available to the listener earlier in time and lower attentional demands compared with AO or VO encoding and retrieval. Significant modality differences in lexical encoding and memory retrieval speeds were observed across modalities. The memory scanning speed advantage observed for AV compared with AO or VO modalities was strongly related to the time course of stimulus information. In contrast, lexical encoding and retrieval speeds for VO words could not be explained by the time-course of stimulus information alone. Working memory processes for the VO modality may be impacted by greater attentional demands and less information availability compared with the AV and AO modalities. Overall, these results support the hypothesis that the presentation modality for speech inputs (AV, AO, or VO) affects how older adult listeners with hearing loss encode, remember, and retrieve what they hear.

The Development of Fantasy Teaching Materials by Using VAK (Visual Auditory Kinesthetic) on 7th Grade Students in SMP Negeri 1 Medan

This study aims to produce teaching materials by using VAK (Visual Auditory Kinestethic) on fantasy story material for 7th grade students of SMP Negeri 1 Medan. The research method used is Research and Development which refers to Borg and Gall model. The stages of development are the initial study phase, initial product development, and product trials. The results showed that the results of material validation included content eligibility with an average of 82% with a very good category, Eligibility for Presentation with an average of 87.5% with a very good category, and an assessment of language aspects with an average of 82.2% with a category very good. For the assessment of graphics by design experts obtained an average of 87% with the category "very good". Product trials are carried out in three stages: individual testing, small group trials, and limited field trials. Individual trials with an average of 86% with very good categories, small group trials with an average of 87.4% with very good categories, and limited field trials with an average of 88.3% with very good categories. The effectiveness of teaching materials is obtained through student learning outcomes in the pretest and posttest. Average score at pretest is 52 and at posttest is 76. This proves that the teaching material of fantasy story by using VAK (Visual Auditory Kinestethic) is feasible, easy, and effective for use in 7th grade students of SMP Negeri 1 Medan. This research has implications for Indonesian language learning process, which is teaching material by using VAK (Visual Auditory Kinestethic) which is developed to provide practical contributions, especially in the implementation of the learning process for teachers, teaching materials in the form of this module can be additional teaching materials to provide convenience in conveying fantasy story material taught, enriching and increasing students' knowledge about fantasy story so that learning will be more interesting and motivate students, trains students to learn independently.

Auditory and auditory-visual frequency-band importance functions for consonant recognition.

The relative importance of individual frequency regions for speech intelligibility has been firmly established for broadband auditory-only (AO) conditions. Yet, speech communication often takes place face-to-face. This study tested the hypothesis that under auditory-visual (AV) conditions, where visual information is redundant with high-frequency auditory cues, lower frequency regions will increase in relative importance compared to AO conditions. Frequency band-importance functions for consonants were measured for eight hearing-impaired and four normal-hearing listeners. Speech was filtered into four 1/3-octave bands each separated by an octave to minimize energetic masking. On each trial, the signal-to-noise ratio (SNR) in each band was selected randomly from a 10-dB range. AO and AV band-importance functions were estimated using three logistic-regression analyses: a primary model relating performance to the four independent SNRs; a control model that also included band-interaction terms; and a different set of four control models, each examining one band at a time. For both listener groups, the relative importance of the low-frequency bands increased under AV conditions, consistent with earlier studies using isolated speech bands. All three analyses showed similar results, indicating the absence of cross-band interactions. These results suggest that accurate prediction of AV speech intelligibility may require different frequency-importance functions than for AO conditions.

Bilateral Cochlear Implants Allow Listeners to Benefit from Visual Information When Talker Location is Varied.

Previous research has found that when the location of a talker was varied and an auditory prompt indicated the location of the talker, the addition of visual information produced a significant and large improvement in speech understanding for listeners with bilateral cochlear implants (CIs) but not with a unilateral CI. Presumably, the sound-source localization ability of the bilateral CI listeners allowed them to orient to the auditory prompt and benefit from visual information for the subsequent target sentence. The goal of this project was to assess the robustness of previous research by using a different test environment, a different CI, different test material, and a different response measure. Nine listeners fit with bilateral CIs were tested in a simulation of a crowded restaurant. Auditory-visual (AV) sentence material was presented from loudspeakers and video monitors at 0, +90, and -90 degrees. Each trial started with the presentation of an auditory alerting phrase from one of the three target loudspeakers followed by an AV target sentence from that loudspeaker/monitor. On each trial, the two nontarget monitors showed the speaker mouthing a different sentence. Sentences were presented in noise in four test conditions: one CI, one CI plus vision, bilateral CIs, and bilateral CIs plus vision. Mean percent words correct for the four test conditions were: one CI, 43%; bilateral CI, 60%; one CI plus vision, 52%; and bilateral CI plus vision, 84%. Visual information did not significantly improve performance in the single CI conditions but did improve performance in the bilateral CI conditions. The magnitude of improvement for two CIs versus one CI in the AV condition was approximately twice that for two CIs versus one CI in the auditory condition. Our results are consistent with previous data showing the large value of bilateral implants in a complex AV listening environment. The results indicate that the value of bilateral CIs for speech understanding is significantly underestimated in standard, auditory-only, single-speaker, test environments.